Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Finding specific series of strings or characters

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Finding specific series of strings or characters
# 8  
Old 10-07-2011
Quote:
Originally Posted by Xterra
I see quite cool! However, I have a couple of problems undertanding your script. First, I do not get anything if I use "/-*/"
probably because that's not what I suggested, try /[-*]/

Quote:
. Second, let suppose I only want to print the sequences that do not contain an "*". Then, I use your code:
Code:
awk 'BEGIN { RS=">"; FS="\n"; OFS="\n"; ORS=">" } !/*/ { if(!P++) printf("%s", RS); print }' infile

This is what I get:
Putting that in quote tags instead of code tags meant it vanished when I quoted it.
Quote:
The script is adding ">" at the very beggining of the file which is wrong
Leave out the if(!P++) printf("%s", RS); then.

Quote:
and the last ">" is retained.
Any suggestions?
Code:
awk 'BEGIN { RS=">"; FS="\n"; OFS="\n"; ORS=">" } $2 && !/[*]/ { print }' infile

Since * is a special character you have to escape it like \* or put it inside a set like[*].
# 9  
Old 10-07-2011
Code:
$0 !~ /[-*]/

means :
$0the current line
!~does not contain
/[-*]/- or *
This User Gave Thanks to ctsgnb For This Post:
Xterra
# 10  
Old 10-07-2011
ctsgnb
Thanks once again!

Corona,
I am still trying to understand your script but I am not able to get it to do exactly what I need. CTSGNB srcipt is working but I am trying to understand the logic behind your script since I believe it might help in the future. So, this is the code:
Code:
awk 'BEGIN { RS=">"; FS="\n"; OFS="\n"; ORS=">" } $0 !~/[-*]/ { print $0 }'

And this is the outfile:
Quote:
>Sequence2
AGACAGATGACAGTAGACAGATAGACGATAGCAGT
>Sequence3
AGACAGATGACAGTAGACAGATCGACGATAGCAGT
>
As you can see I still have the ">" at the end of the file which completely messes up the FASTA format. I have been trying to get rid of it by modifying your script but I just cannot get the job done. Can you help me one more time?
Thanks!
# 11  
Old 10-07-2011
If awk's default handling of ORS doesn't do what you want, you'll have to print the >'s yourself:

Code:
$ cat data
>Sequence1
AGACAGATGACAGTAGACAGAT-GACGATAGCAGT
>Sequence2
AGACAGATGACAGTAGACAGATAGACGATAGCAGT
>Sequence3
AGACAGATGACAGTAGACAGATCGACGATAGCAGT
>Sequence4
AGACAGATGA-AGTAGACAGATTGACGATAGCAGT
>Sequence5
AGAC*GATGA
$ awk 'BEGIN { RS=">"; FS="\n"; ORS="" } /-/ { print ">" $0; }' data
>Sequence1
AGACAGATGACAGTAGACAGAT-GACGATAGCAGT
>Sequence4
AGACAGATGA-AGTAGACAGATTGACGATAGCAGT
$

[edit] adding a reply that explains in more detail.

---------- Post updated at 12:15 PM ---------- Previous update was at 12:05 PM ----------

You know how the FS and OFS variables control what awk considers fields for input, and what awk prints as fields for output?

RS and ORS are the exact same thing, but for lines. So when we do RS=">"; FS="\n" we're telling awk "each time you see >, that is a new line", and "each time you see \n, that's a new field".

When you have a statement like
Code:
EXPRESSION { code }

, the { code } part is only executed when EXPRESSION is true. If you drop an unadorned /regex/ into there, it assumes you want $0 ~ /regex/. BEGIN and END are just special expressions that are true before any processing, and after all records have been processed.

My first try puts extra >'s on the end because the record separator gets printed at the end of the record, not the beginning -- the same place you'd expect a newline. So it ends up kind of off by one.

My improved version here just prepends a > to the input string and prints it, so it gets them in the correct place.

So:

Code:
BEGIN {
        # Our 'newline' will be >
        RS=">";
        # Input fields separated on real newlines
        FS="\n";
} 

# This code block gets executed only when $0 ~ /-/
# i.e. there's a - somewhere in the entire mess of input for this 'line'.
# If you wanted to just check the second field, you could do
# $2 ~ /-/ { ... }
/-/ {
        # Print a >, followed by all our fields.  Since we haven't
        # modified $1/$2/..., $0 will still contain UNMODIFIED data,
        # complete with newlines -- otherwise we might need OFS="\n"
        # to print newlines instead of spaces between lines.
        print ">" $0;
}

Last edited by Corona688; 10-07-2011 at 03:26 PM..
# 12  
Old 10-07-2011
I see!
However, adding "!" to the script changes the output slightly:
Code:
awk 'BEGIN { RS=">"; FS="\n"; ORS="" } !/[-*]/ { print ">" $0; }'

Output:
Quote:
>>Sequence2
AGACAGATGACAGTAGACAGATAGACGATAGCAGT
>Sequence3
AGACAGATGACAGTAGACAGATCGACGATAGCAGT
It does not affect the format of the file though.
# 13  
Old 10-07-2011
On of the way to get read off the trailing ">" is to pre-format your output that way :

Code:
nawk 'NR>1&&/^>/{sub(">",RS ">")}1' yourfile

You can then pipe your output and parse it using RS="" (i.e. "\n\n") as record separator :

Code:
nawk 'NR>1&&/^>/{sub(">",RS ">")}1' yourfile | nawk 'BEGIN{RS="";FS="\n"}$0!~/[-*]/'

Code:
$ cat f1
>Sequence1
AGACAGATGACAGTAGACAGAT-GACGATAGCAGT
>Sequence2
AGACAGATGACAGTAGACAGATAGACGATAGCAGT
>Sequence3
AGACAGATGACAGTAGACAGATCGACGATAGCAGT
>Sequence4
AGACAGATGA-AGTAGACAGATTGACGATAGCAGT
>Sequence5
AGAC*GATGA

Code:
$ nawk 'NR>1&&/^>/{sub(">",RS ">")}1' f1
>Sequence1
AGACAGATGACAGTAGACAGAT-GACGATAGCAGT

>Sequence2
AGACAGATGACAGTAGACAGATAGACGATAGCAGT

>Sequence3
AGACAGATGACAGTAGACAGATCGACGATAGCAGT

>Sequence4
AGACAGATGA-AGTAGACAGATTGACGATAGCAGT

>Sequence5
AGAC*GATGA

Code:
$ nawk 'NR>1&&/^>/{sub(">",RS ">")}1' f1 | nawk 'BEGIN{RS="";FS="\n"}$0!~/[-*]/'
>Sequence2
AGACAGATGACAGTAGACAGATAGACGATAGCAGT
>Sequence3
AGACAGATGACAGTAGACAGATCGACGATAGCAGT

Last edited by ctsgnb; 10-07-2011 at 03:41 PM..
# 14  
Old 10-07-2011
Quote:
Originally Posted by Xterra
However, adding "!" to the script changes the output slightly:
You might have blank lines at the start of the file. It sees
Code:
( blank lines)

>stuff
GTCATC

as the first record and, since it contains no -, happily prints it.

Either that, or your version of awk is quite happy to believe that > at the beginning of the file implies a completely blank record before it. Mine doesn't, but an easy fix anyway -- just tell it not to print the first record.

Code:
awk 'BEGIN { RS=">"; FS="\n"; ORS="" } ( (NR>1) && !/[-*]/ ) { print ">" $0; }'

Last edited by Corona688; 10-07-2011 at 03:49 PM.. Reason: many edits, hopefully not stealth ones.
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Printing lines with specific strings at specific columns

Hi I have a file which is tab-delimited. Now, I'd like to print the lines which have "chr6" string in both first and second columns. Could anybody help? (3 Replies)
Discussion started by: a_bahreini
3 Replies

2. Shell Programming and Scripting

Finding Strings between 2 characters in a file

Hi All, Assuming i have got a file test.dat which has contains as follows: Unix = abc def fgt jug 111 2222 3333 Linux = gggg pppp qqq C# = ccc ffff llll I would like to traverse through the file, get the 1st occurance of "=" and then need to get the sting... (22 Replies)
Discussion started by: rtagarra
22 Replies

3. Shell Programming and Scripting

Count specific characters at specific column positions

Hi all, I need help. I have an input text file (input.txt) like this: 21 GTGCAACACCGTCTTGAGAGG 50 21 GACCGAGACAGAATGAAAATC 73 21 CGGGTCTGTAGTAGCAAACGC 108 21 CGAAAAATGAACCCCTTTATC 220 21 CGTGATCCTGTTGAAGGGTCG 259 Now I need to count A/T/G/C numbers at each character location in column... (2 Replies)
Discussion started by: thienxho
2 Replies

4. Shell Programming and Scripting

Can't figure out how to find specific characters in specific columns

I am trying to find a specific set of characters in a long file. I only want to find the characters in column 265 for 4 bytes. Is there a search for that? I tried cut but couldn't get it to work. Ex. I want to find '9999' in column 265 for 4 bytes. If it is in there, I want it to print... (12 Replies)
Discussion started by: Drenhead
12 Replies

5. Shell Programming and Scripting

finding the strings beween 2 characters "/" & "/" in .txt file

Hi all. I have a .txt file that I need to sort it My file is like: 1- 88 chain0 MASTER (FF-TE) FFFF 1962510 /TCK T FD2TQHVTT1 /jtagc/jtag_instreg/updateinstr_reg_1 dff1 (TI,SO) 2- ... (10 Replies)
Discussion started by: Behrouzx77
10 Replies

6. Shell Programming and Scripting

sed replacing specific characters and control characters by escaping

sed -e "s// /g" old.txt > new.txt While I do know some control characters need to be escaped, can normal characters also be escaped and still work the same way? Basically I do not know all control characters that have a special meaning, for example, ?, ., % have a meaning and have to be escaped... (11 Replies)
Discussion started by: ijustneeda
11 Replies

7. Shell Programming and Scripting

Finding Minimum in a Series

I have two LARGE files of data more than 20,000 line each, file-1 and file-2, and I wish to do the following if possible: file-1 1 2 5 7 9 2 4 6 3 8 9 4 6 8 9 3 2 1 3 1 2 . . . file-2 1 2 3 2 5 7 5 7 3 7 9 4 . (5 Replies)
Discussion started by: ali2011
5 Replies

8. Shell Programming and Scripting

Finding repitition of series

Dear friends, hello to everyone. I am new to this forum. I have a set of data where I need to find the repitition of series as below data format: 0001230000456000001230000456 each digit can be separated by any delimeter I need to find out the starting point (index) of '123' and '456' I... (2 Replies)
Discussion started by: gjarms
2 Replies

9. Shell Programming and Scripting

Finding strings

Hi I made a post earlier but now my problem has become a lot more complicated. So I have a file that looks like this: Name 1 13 94 1 AGGTT Name 1 31 44 1 TTCCG Name 1 13 94 2 AAAAATTTT Name 1 41 47 2 GGGGGGGGGGG So the file is tab delimited and what I want to do is find... (8 Replies)
Discussion started by: kylle345
8 Replies

10. Shell Programming and Scripting

print 10 characters in series

suppose fileA kanika123ABC 1222222222222222 raciat5678ty 1221123333331121 jessica78ulllo 2233243223333333 so output shud be print only first 10 characters in series and rest remain same kanika123A 1222222222222222 raciat5678 1221123333331121 jessica78u ... (1 Reply)
Discussion started by: cdfd123
1 Replies
Login or Register to Ask a Question