Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Extract regular expression and line below

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract regular expression and line below
# 1  
Old 08-25-2012
Extract regular expression and line below
Hi all, I have a large fasta (dna sequence) file. I would like to extract a portion of the header as well as the sequence (line below the header).

Input:
Quote:
>gi|299507456|gb|GL349621.1| Acyrthosiphon pisum unplaced genomic scaffold Scaffold1, whole genome shotgun sequence
TTTACAATTGCTATTGTAACAATATATCAGGAGCCTTGTATTAAATTTTCACGCATTTTTACCAAACAAATAAAATTTTATTGAT
Output:
Quote:
>GL349621.1
TTTACAATTGCTATTGTAACAATATATCAGGAGCCTTGTATTAAATTTTCACGCATTTTTACCAAACAAATAAAATTTTATTGAT
All accession values (the term I want to preserve, which is the string including and directly following "GL") are different, but I believe they are the same length.

I'm a command-line beginning. I tried to adapt code I found online, but though it does preserve the sequence line, it only cuts off the portion of the header following the accession, not before it.
Code:
sed 's/| [^ ].* *//g'

I have also tried:
Code:
grep -o 'GL\d{6}\.1'

but it also doesn't work.

Any suggestions?
# 2  
Old 08-25-2012
Assuming that all of the file consists of the two records (alternating) that you have posted, then this will print the portion of the first record you've pointed out, and the entire next record.

If there are other records in the file this won't work.

Code:
awk -F \| ' { printf( ">%s\n", $4 ); getline;  print; }'  input-file >output-file

Last edited by agama; 08-25-2012 at 08:43 PM.. Reason: fixed missing semicolon
# 3  
Old 08-25-2012
Each record is different. Here is a slightly expanded excerpt from the file:

Quote:
>gi|299507456|gb|GL349621.1| Acyrthosiphon pisum unplaced genomic scaffold Scaffold1, whole genome shotgun sequence
TTTACAATTGCTATTGTAACAATATATCAGGAGCCTTGTATTAAATTTTCACGCATTTTTACCAAACAAATAAAATTTTATTGAT
>gi|299507455|gb|GL349622.1| Acyrthosiphon pisum unplaced genomic scaffold Scaffold2, whole genome shotgun sequence
GTATGCGCGCATCTCCATACCGTCCGATAAATTCGCAGTAAAAAAAATGTGATTCACATTGTCGATTATAATAAAAAAAT
>gi|299507454|gb|GL349623.1| Acyrthosiphon pisum unplaced genomic scaffold Scaffold3, whole genome shotgun sequence
AATATTAAATAATTAATCTAAATAAATTAAATACCTCATTAGTCATTAACACACATTTTTTTCTTAGTTTTAATGTATAA
# 4  
Old 08-25-2012
The awk I posted should work, a bit of a tweak (not necessary I don't think)

Code:
awk -F \| ' /^>gi/  { printf( ">%s\n", $4 ); getline;  print; }'  input-file >output-file

By the same I meant same format. Record a separated by pipe symbols (|) followed by record b (no implied format).
Last edited by agama; 08-25-2012 at 08:42 PM.. Reason: fixed missing semicolon
# 5  
Old 08-25-2012
Thanks, agama, but with the following command I get the following error:
Code:
awk -F \| ' /^>gi/ { printf( ">%s\n", $4 ); getline print; }' test3.fa

awk: syntax error at source line 1
context is
/^>gi/ { printf( ">%s\n", $4 ); getline >>> print <<< ; }
awk: illegal statement at source line 1

Code:
cat test3.fa 
>gi|299507456|gb|GL349621.1| Acyrthosiphon pisum unplaced genomic scaffold Scaffold1, whole genome shotgun sequence
TTTACAATTGCTATTGTAACAATATATCAGGAGCCTTGTATTAAATTTTCACGCATTTTTACCAAACAAATAAAATTTTATTGAT
>gi|299507455|gb|GL349622.1| Acyrthosiphon pisum unplaced genomic scaffold Scaffold2, whole genome shotgun sequence
GTATGCGCGCATCTCCATACCGTCCGATAAATTCGCAGTAAAAAAAATGTGATTCACATTGTCGATTATAATAAAAAAAT
>gi|299507454|gb|GL349623.1| Acyrthosiphon pisum unplaced genomic scaffold Scaffold3, whole genome shotgun sequence
AATATTAAATAATTAATCTAAATAAATTAAATACCTCATTAGTCATTAACACACATTTTTTTCTTAGTTTTAATGTAT

# 6  
Old 08-25-2012
Sorry -- I tested it as a multi-line programme, but joined the lines when I posted it and forgot a semicolon.

Code:
awk -F \| ' /^>gi/ { printf( ">%s\n", $4 ); getline; print; }' test3.fa

This User Gave Thanks to agama For This Post:
pathunkathunk
# 7  
Old 08-26-2012
Perl

Code:
perl -nle 'if ($flg){print;$flg=0}if(/^\>gi/){ print ">",((split(/\|/))[3]);$flg=1}' File1
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

regular expression with shell script to extract data out of a text file

hi i am trying to extract some specific data out of a text file using regular expressions with shell script that is using a multiline grep .. and the tool i am using is pcregrep so that i can get compatibility with perl's regular expressions for a sample data like this, i am trying to grab... (6 Replies)
Discussion started by: vemkiran
6 Replies

2. Programming

Perl: How to read from a file, do regular expression and then replace the found regular expression

Hi all, How am I read a file, find the match regular expression and overwrite to the same files. open DESTINATION_FILE, "<tmptravl.dat" or die "tmptravl.dat"; open NEW_DESTINATION_FILE, ">new_tmptravl.dat" or die "new_tmptravl.dat"; while (<DESTINATION_FILE>) { # print... (1 Reply)
Discussion started by: jessy83
1 Replies

3. Shell Programming and Scripting

Regular Expression for line

Hi, I'm trying to extract if the (offset>0 || flags ) from the following line: 90.30.180.90.80 > 90.121.333.308.45133: Flags , seq 14480:15928, ack 1, win 8088, options > 15:59:11.156664 IP (tos 0x0, ttl 20, id 44442, offset 0, flags , proto TCP (6), length 1500) try: offset.(+) ||... (7 Replies)
Discussion started by: ENG_MOHD
7 Replies

4. Shell Programming and Scripting

Regular expression to extract ipv6 address

Hi all , I have a string in my weblog xheader v6-day-2011:xx:yy:zz:qq:qq:ww:ee:rr My requirement is to lookup the sting v6-day-2011 in this header and if found would like to extract the V6 ip part . v6-day-2011 is always constant for a ipv6 entry so i would like to extract every thing... (4 Replies)
Discussion started by: jambesh
4 Replies

5. Shell Programming and Scripting

Grep regular expression to get part of a line

Hi I just started on GNU Grep with regex and am finding it very challenging and need to ask for help already... here is the problem, I have a page (MYFILE) which consists of the following.... <div> <input type="hidden" name="__EVENTTARGET" id="__EVENTTARGET" value="" /> <input type="hidden"... (2 Replies)
Discussion started by: noobie74645
2 Replies

6. Shell Programming and Scripting

regular expression format string in one line.

Hi All, @months = qw(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec); $day=091023; $day_combine = $day; $day_combine =~ s/({2})({2})({2})/20$1-$months-$3/; Instead of three lines, is possible to combine the last two lines into a single line? means no need assign $day to $day_combine... (2 Replies)
Discussion started by: jimmy_y
2 Replies

7. Shell Programming and Scripting

New line problem of regular expression

could anybody tell me how i can add/append a new line using regular expression in vi on AIX? i've tried several ways before, but all of them failed. e.g. :%s/$/\n/ :%s/^/\v\r/ :( (1 Reply)
Discussion started by: wrl
1 Replies

8. Linux

Regular expression to extract "y" from "abc/x.y.z" .... i need regular expression

Regular expression to extract "y" from "abc/x.y.z" (2 Replies)
Discussion started by: rag84dec
2 Replies

9. Shell Programming and Scripting

Extract a substring using regular expression

Hello: I'm trying to extracta a matching substring from a string using regular expression. I need to extract the date part of any giving string. All input string will have date in YYYYMMDD format in them, but it can be anywhere in the string. Eg. The_Mummy20080125_New... (2 Replies)
Discussion started by: apraja
2 Replies

10. Shell Programming and Scripting

Regular expression matching a new line

I have written a script to test some isdn links in my network and I am trying to format the output to be more readable. Each line of the output has a different number of digits as follows... Sitename , spid1 12345678901234 1234567890 1234567 , spid2 1234567890 1234567890 1234567 Sitename , ... (1 Reply)
Discussion started by: drheams
1 Replies
Login or Register to Ask a Question