Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Filter (by max length) only lines not matching regex

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Filter (by max length) only lines not matching regex
# 1  
Old 03-25-2013
Filter (by max length) only lines not matching regex
I have a large file of many pairs of sequences and their headers, which always begin with '>'
Quote:
>gi|abc| Species one
ATCAGGTATAATT
>gi|bcd| Species two
ATTTGATC
>gi|cdf| Species three
ATTTGATCT
I'm looking for help on how to retain only sequences (and their headers) below a certain length. So if min length was 10, output would be
Quote:
>gi|bcd| Species two
ATTTGATC
>gi|cdf| Species three
ATTTGATCT
I can filter by length, but I'm not sure how to exclude the header lines.
Code:
awk '{lines[NR] = $0} length($0) < 10 {print lines [NR-1]; print lines [NR]} ' file.name

# 2  
Old 03-25-2013
Do your data always come in pairs? If yes, your code is fine. Try also
Code:
$ while read HEAD; do read DATA; [ ${#DATA} -lt 10 ] && printf "%s\n%s\n" "$HEAD" "$DATA"; done < file
>gi|bcd| Species two
ATTTGATC
>gi|cdf| Species three
ATTTGATCT

This User Gave Thanks to RudiC For This Post:
pathunkathunk
# 3  
Old 03-25-2013
Like this?
Code:
awk '/^[^>]/ && length($0)<10{print hdr"\n"$0}{hdr=$0}' input

This User Gave Thanks to mirni For This Post:
pathunkathunk
# 4  
Old 03-25-2013
Both of these work, thank you.

The problem with my original code was obscured by my choice of an example file with unrealistically short sequences. In reality my sequences are longer, and the problem with my code is that it captures headers along with short sequences.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Filter Row Based On Max Column Value After Group BY

Hello Team, Need your expertise on following: Here is the set of data: C1|4|C1SP1|A1|C1BP1|T1 C1|4|C1SP2|A1|C1BP2|T2 C2|3|C2SP1|A2|C2BP1|T2 C3|3|C3SP1|A3|C3BP1|T2 C2|2|C2SP2|A2|C2BP2|T1 I need to filter above date base on following two steps: 1. Group them by column 1 and 4 2.... (12 Replies)
Discussion started by: angshuman
12 Replies

2. Shell Programming and Scripting

Filter all the lines with minimum specified length of words of a text file

Hi Can someone tell me which script will work best (in terms of speed and simplicity to write and run) for a large text file to filter all the lines with a minimum specified length of words ? A sample script with be definitely of great help !!! Thanks in advance. :) (4 Replies)
Discussion started by: my_Perl
4 Replies

3. UNIX for Dummies Questions & Answers

awk loop for to filter lines by max value

Hi all, I'm struggling to filter my data frame. I need to print only those lines whose max value (the number of columns may vary) is above a cut-off value. My data looks like this: chr22 17565753 17565754 5 4 5 5 6 2 5 5 6 2 chr22 17565754 ... (2 Replies)
Discussion started by: lsantome
2 Replies

4. Shell Programming and Scripting

Filter file by length, looking only at lines that don't begin with ">"

I have a file that stores data in pairs of lines, following this format: line 1: header (preceded by ">") line 2: sequence Example.txt: >seq1 name GATTGATGTTTGAGTTTTGGTTTTT >seq2 name TTTTCTTC I want to filter out the sequences and corresponding headers for all sequences that are less... (2 Replies)
Discussion started by: pathunkathunk
2 Replies

5. Shell Programming and Scripting

Perl Regex matching multiple lines

I need a way to extract data from X 4T Solution 21 OCT 2011 37 .00 to account 12345678 User1 user2 X 4T Solution Solution Unlimited 11 Sep 2009 248 .00 to account 87654321 user3 user4 I need it to extract 'X' '37.00' and account number 12345678. I have extracted above stuff... (3 Replies)
Discussion started by: chakrapani
3 Replies

6. UNIX for Dummies Questions & Answers

Modify the max username length

Hey Any one... Do u know any way I can modify the max username length in unix? I guess it is 32/64 characters by default. Suppose I want to increase it to 128. i hav tried /etc/skel but no use... How can I do that? (2 Replies)
Discussion started by: MayureshRisbud
2 Replies

7. UNIX for Advanced & Expert Users

How to increase max username length?

Hi, This is my first post to this site. So kindly forgive if I am writing in a wrong section. My query is that... I want to modify the max username length size. I guess it is 32/64 on CentOS. Now I want to change it to 128. Is there any way to do that? Thanks in advance!! :) (4 Replies)
Discussion started by: ajay303
4 Replies

8. Shell Programming and Scripting

Counting the max length of string

Hi all, I have a flat file of 1000 rows. I want to check the length of the 5th column. The one having the longest length , I want to set it as DEFINED PARAMETER. So later I can check others with that particular number only. Any ideas ?? (2 Replies)
Discussion started by: ganesh123
2 Replies

9. UNIX for Dummies Questions & Answers

Length of a Unix filepath max length

Hi Guys, Could anyone shed some light on the length of a Unix filepath max length pls ? thanks ! Wilson (3 Replies)
Discussion started by: wilsontan
3 Replies

10. UNIX for Dummies Questions & Answers

Sed working on lines of small length and not large length

Hi , I have a peculiar case, where my sed command is working on a file which contains lines of small length. sed "s/XYZ:1/XYZ:3/g" abc.txt > xyz.txt when abc.txt contains lines of small length(currently around 80 chars) , this sed command is working fine. when abc.txt contains lines of... (3 Replies)
Discussion started by: thanuman
3 Replies
Login or Register to Ask a Question