Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Extract distinc sequence of letters

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Extract distinc sequence of letters
# 1  
Old 10-14-2014
Extract distinc sequence of letters
Hallo,
I need to extract distinct sequence of letters for example from 136 to 193
Files are quite big, so I would prefer not to use "fold -w1"
Thank you very much

Input file look like this:
Code:
       1 cttttacctt catgtgtttt tgcagatatt tgttcataat aacatcttct ttttaagtta
      61 ttaaaatctt ttttaaagtt attaacattt ttttgtcttt tagatcctat atagatccta
      121 aaagatccta aaagatccta aaagatcccc gtttttgtta aagcatatgt gataaggttt
      181 tatagtactt taagattcac tatagtcagt aaaacgttca ctatagtcag taaaacgttc

Last edited by Don Cragun; 10-14-2014 at 04:00 AM.. Reason: Add CODE tags.
# 2  
Old 10-14-2014
Quote:
Originally Posted by kamcamonty
Hallo,
I need to extract distinct sequence of letters for example from 136 to 193
Files are quite big, so I would prefer not to use "fold -w1"
Thank you very much

Input file look like this:
Code:
       1 cttttacctt catgtgtttt tgcagatatt tgttcataat aacatcttct ttttaagtta
      61 ttaaaatctt ttttaaagtt attaacattt ttttgtcttt tagatcctat atagatccta
      121 aaagatccta aaagatccta aaagatcccc gtttttgtta aagcatatgt gataaggttt
      181 tatagtactt taagattcac tatagtcagt aaaacgttca ctatagtcag taaaacgttc

I'm lost.
  1. Please use CODE tags.
  2. From 136 to 193 what? You have shown us lines with a variable length digit string followed by 5 or 6 groups of 10 letters. The only thing you have shown us between 136 and 193 is the digit string 181 marked in red above.
  3. What do you mean by "Files are quite big"? How long will the longest lines be in your input files? What is the maximum size (in bytes) of your input files?
  4. What output are you expecting from the above sample input?
  5. I understand that you don't want to use fold -w1, but I don't understand why you would say that. I don't see how using fold -w1 to put each character in your input files on a single line would help solve this problem. (But, maybe that is just because I can't figure out what you're trying to do.)
  6. What OS and shell are you using?
Last edited by Don Cragun; 10-14-2014 at 04:48 AM.. Reason: Add request for OS and shell information.
# 3  
Old 10-14-2014
2. Input file is in fact one sequence of letters which is separated into lines and numbered (and each lines is separated into sequences of 10 by space). If it would be better I can create one long line from whole input file at first. I want to create shorter sequence from each file such that Nth letter (eg.: third or 136th) of the sequence will be first letter of new sequence and Mth letter (e.g.: 196th) is the last one. (Just imagine all letters are numbered and I want all letters which has numbers greater or equal than 136 and smaller than 196)
I use zsh but is no problem to use bash; OS: Biolinux (Ubuntu)
3. All lines are of this length (just length of number is variable), but there can be about 1 000 000 lines in each file
5. I wanted to a) remove spaces and numbers b) put each character on new line c) select lines containing characters I wanted using awk NR d) join all lines into one
# 4  
Old 10-14-2014
Still not clear. If you want to remove the line count, all white space including new lines, and then extract some chars, this might help:
Code:
awk '{$1=""; gsub(/ /,"")}1' ORS="" file | dd bs=1 skip=135 count=58 2>/dev/null
tcctaaaagatccccgtttttgttaaagcatatgtgataaggttttatagtactttaa

This works for the sample you gave us; not sure how it behaves on large files.
# 5  
Old 10-14-2014
You could also try something like:
Code:
#!/bin/ksh
IAm=${0##*/}
if [ $# -lt 3 ]
then	printf 'Usage: %s first_char# last_char# file...\n' "$IAm" >&2
	exit 1
fi
first="$1"
last="$2"
shift 2
awk -v fc="$first" -v lc="$last" '
BEGIN {	fl = int((fc - 1) / 60) + 1	# first line # containing data to copy
	ll = int((lc - 1) / 60) + 1	# last line # containing data to copy
	flc1 = fc % 60 ? fc % 60 : 60	# first character # to copy on line fl
	llcl = lc % 60 ? lc % 60 : 60	# last character # to copy on line ll
}
FNR >= fl && FNR <= ll {
	s = $2 $3 $4 $5 $6 $7
	printf("%s%s", substr(s, FNR == fl ? flc1 : 1,
		FNR == ll ? FNR == fl ? llcl - flc1 + 1 : llcl : 60),
		FNR == ll ? "\n" : "")
	if(FNR == ll) nextfile
}' "$@"

I prefer ksh over bash, but this script will work with either shell. This script allows you to specify the starting character number, the last character number, and a list of one or more files to process. It should work fine on any Linux system, but the awk nextfile command is an extension to the standards. If your version of awk does not have nextfile:
  • if you only want to process one file at a time, change nextfile to exit,
  • otherwise, remove the entire line shown in red (it will still produce correct output, but will run slower; especially on large files). Note that the code shown in blue can be removed as long as this line remains in your code (with either exit or nextfile.)

If someone else reading this thread wants to try this on a Solaris/SunOS system, change awk in the script to /usr/xpg4/bin/awk, /usr/xpg6/bin/awk, or /usr/bin/nawk.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Random letters

Hi there, first of all this is not homework...this is a new type of exercise for practicing vocabulary with my students. I have a file consisting of two columns, separated by a tab, each line consisting of a word and its definition, separated by a line break. What i need is to replace a... (15 Replies)
Discussion started by: eldeingles
15 Replies

2. Shell Programming and Scripting

Extract sequence from fasta file

Hi, I want to match the sequence id (sub-string of line starting with '>' and extract the information upto next '>' line ). Please help . input > fefrwefrwef X900 AGAGGGAATTGG AGGGGCCTGGAG GGTTCTCTTC > fefrwefrwef X932 AGAGGGAATTGG AGGAGGTGGAG GGTTCTCTTC > fefrwefrwef X937... (2 Replies)
Discussion started by: ritakadm
2 Replies

3. UNIX for Dummies Questions & Answers

sed - extract a group of Letters/numbers

I have a file with hundreds of lines in it. I wanted to extract anything that matches the following: KR followed by 4 digits: example KR1201 cat list | sed "s///g" Is the closest I've come, and obviously it is not what I want. This would remove all of the items that I want and leave me... (2 Replies)
Discussion started by: newbie2010
2 Replies

4. Solaris

Escape Sequence for Capital Letters Input at Shell Not Working

Hello, I am running Solaris 8. When issuing the command "stty lcase" all text which is output to the terminal are capitalized. Letters that are supposed to be capitals are preceded by a backslash during output. All text which is input is converted to lower case. This is the expected behaviour... (5 Replies)
Discussion started by: rstor
5 Replies

5. Shell Programming and Scripting

find common entries and match the number with long sequence and cut that sequence in output

Hi all, I have a file like this ID 3BP5L_HUMAN Reviewed; 393 AA. AC Q7L8J4; Q96FI5; Q9BQH8; Q9C0E3; DT 05-FEB-2008, integrated into UniProtKB/Swiss-Prot. DT 05-JUL-2004, sequence version 1. DT 05-SEP-2012, entry version 71. FT COILED 59 140 ... (1 Reply)
Discussion started by: manigrover
1 Replies

6. Shell Programming and Scripting

Randomize letters

Hi, Is there a tool somewhat parallel to rev, but which randomizes instead of reverses? I've tried rl, but I can only get it to randomize words. I was hoping for something like this echo "hello" | ran leolh less simpler solutions are also welcome. Sorry if the question is... (21 Replies)
Discussion started by: jeppe83
21 Replies

7. Shell Programming and Scripting

Extract sequence blocks

Hi, I have an one-line file consisting of a sequence of 660 letters. I would like to extract 9-letter blocks iteratively: ASDFGHJKLQWERTYUIOPZXCVBNM first block: ASDFGHJKL 1nd block: SDFGHJKLQ What I have so far only gives me the first block, can anyone please explain why? cat... (7 Replies)
Discussion started by: solli
7 Replies

8. Shell Programming and Scripting

extract words with even numbr of letters

Hello All I need to extract words which are of even number of letters and not greater than 10. Any help?? Thanks, Manish (3 Replies)
Discussion started by: manish205
3 Replies

9. Shell Programming and Scripting

Extract Pattern Sequence

Dear Collegues I have to extract Some pattern from raw text file using perl The input will be raw text. Pattern to get - Sequence of Capital Letter Words ( e.g. he is working in Center for Perl Studies. He will come tomorrow...) from thos I have to extract sequences like "Center for Perl... (5 Replies)
Discussion started by: jaganadh
5 Replies

10. Shell Programming and Scripting

How to extract a sequence of n lines from a file

Hi I want to be able to extract a sequence of n lines from a file. ideas, commands and suggestions would be highly appreciated. Thanks (4 Replies)
Discussion started by: 0ktalmagik
4 Replies
Login or Register to Ask a Question