Shell Programming and Scripting

Hi all,
Im stuck in findind solution to ths problem. Please guide me if u have any ideas.

I have two files.

===FILE1===
>bi|2138271|geb|AAC15885.1|precursor [Sambucus nigra]
MRVIAAAMLYLYIVVLAICSVGIQGIDYPSVSFNLAGAKSATWDFLRMPHDLVGEDNKYNDGEPITGNII
GRDGLCVDVRNGYDTDGTPLQLWPCGTQRNQQWTFYTDDTIRSMGKCMTANGLSNGSNIMIFNCSTAVEN
AIKWEVTIDGSIINPSSG
>bi|21083|em|CAA26939.1| precursor [Ricinus communis]
MKPGGNTIVIWMYAVATWLCFGSTSGWSFTLEDNNIFPKQYPIINFTTAGATVQSYTNFIRAVRGRLTTG
ADVRHEIPVLPNRVGLPINQRFILVELSNHAELSVTLALDVTNAYVVGYRAGNSAYFFHPDNQEDAEAIT
HLFTDVQNRYTFAFGGNYDRLEQLAGNLRENIELGNGPLEEAISALYYYSTGGTQLPTL
>bi|19526601|geb|AAL87006.1| chain A [Viscum album]
YERLRLRVTHQTTGEEYFRFITLLRDYVSSGSFSNEIPLLRQSTIPVSDAQRFVLVELTNEGGDSITAAI
DVTNLYVVAYQAGDQSYFLRDAPRGAETHLFTGTTRSSLPFNGSYPDLERYAGHRDQIPLGIDQLIQSVT
ALRFPGGNTRTQARSILILIQMISEAARFNPILWRARQYINSGASFLPDVY

=====FILE2====
bi|2138271|geb|AAC15885----- 20 ---- 32
bi|2138271|geb|AAC15885 ----- 92 ---- 110
bi|19526601|geb|AAL8700 ----- 74 ---- 92
bi|2138271|geb|AAC15885 ----- 26 ---- 38
bi|21083|em|CAA26939.1| ----- 19 ---- 37
bi|21083|em|CAA26939.1| ----- 52 ---- 70

please note that ----- is not present in actual file i just wrote to make the columns visible otherwise it is a tab separated file.

It has three columns-
column1 contains patterns from FILE1
column 2 contains the start position to cut the string
and column3 contains end position til the string has to be cut
WHERE POSITION 1 STARTS FROM THE LINE CONTAINING ALL THE ALPHABETS IN CAPS WITHOUT ANY SPACE AND ENDS WHERE NEXT '>' SYMBOL APPEARS

I want to have a new file that contains header(LINE BEGINING WITH '>' SYMBOL) maching first column of FILE2 and then only a substring

for eg- in case of first record of FILE2

The ouput file should contain-
>bi|2138271|geb|AAC15885.1|precursor [Sambucus nigra] (20-32)
SVGIQGIDYPSV

and so on------ for all the other records of FILE2

Your contribution is highly appretiated.
Thanks
Smriti

I am not sure that the output of the following script is what what you are looking for.
Try and adapt it.

Code:

awk -F'\t' '

NR==FNR {
   if (NR==1)
      Key_len = length($1) + 1;
   k = ">" substr($0,1, Key_len-1);
   n = ++Keys[k];
   From[k,n] = $2;
     To[k,n] = $3;
    Len[k,n] = $3 - $2;
   next;
}

function print_selected(    i,k) {
   if (selected) {
      k = substr(Header, 1, Key_len);
      for (i=1; i<=Keys[k]; i++) {
         printf("%s (%s-%s)\n", Header, From[k,i], To[k,i]);
         print substr(Alphabets, From[k,i], Len[k,i]);
      }
   }
}

/^>/ {
   print_selected();
   selected  = (substr($0, 1, Key_len) in Keys);
   Header    = $0;
   Alphabets = "";
   next;
}

selected {
   Alphabets = Alphabets $0;
}

END {
   print_selected();
}
' FILE2 FILE1

Input file FIL1 :

Code:

>bi|2138271|geb|AAC15885.1|precursor [Sambucus nigra]
MRVIAAAMLYLYIVVLAICSVGIQGIDYPSVSFNLAGAKSATWDFLRMPHDLVGEDNKYNDGEPITGNII
GRDGLCVDVRNGYDTDGTPLQLWPCGTQRNQQWTFYTDDTIRSMGKCMTANGLSNGSNIMIFNCSTAVEN
AIKWEVTIDGSIINPSSG
>bi|21083|em|CAA26939.1| precursor [Ricinus communis]
MKPGGNTIVIWMYAVATWLCFGSTSGWSFTLEDNNIFPKQYPIINFTTAGATVQSYTNFIRAVRGRLTTG
ADVRHEIPVLPNRVGLPINQRFILVELSNHAELSVTLALDVTNAYVVGYRAGNSAYFFHPDNQEDAEAIT
HLFTDVQNRYTFAFGGNYDRLEQLAGNLRENIELGNGPLEEAISALYYYSTGGTQLPTL
>bi|19526601|geb|AAL87006.1| chain A [Viscum album]
YERLRLRVTHQTTGEEYFRFITLLRDYVSSGSFSNEIPLLRQSTIPVSDAQRFVLVELTNEGGDSITAAI
DVTNLYVVAYQAGDQSYFLRDAPRGAETHLFTGTTRSSLPFNGSYPDLERYAGHRDQIPLGIDQLIQSVT
ALRFPGGNTRTQARSILILIQMISEAARFNPILWRARQYINSGASFLPDVY

Input file FILE2 :

Code:

bi|2138271|geb|AAC15885 20      32
bi|2138271|geb|AAC15885 92      110
bi|19526601|geb|AAL8700 74      92
bi|2138271|geb|AAC15885 26      38
bi|21083|em|CAA26939.1| 19      37
bi|21083|em|CAA26939.1| 52      70

Output:

Code:

>bi|2138271|geb|AAC15885.1|precursor [Sambucus nigra] (20-32)
SVGIQGIDYPSV
>bi|2138271|geb|AAC15885.1|precursor [Sambucus nigra] (92-110)
LWPCGTQRNQQWTFYTDD
>bi|2138271|geb|AAC15885.1|precursor [Sambucus nigra] (26-38)
IDYPSVSFNLAG
>bi|21083|em|CAA26939.1| precursor [Ricinus communis] (19-37)
LCFGSTSGWSFTLEDNNI
>bi|21083|em|CAA26939.1| precursor [Ricinus communis] (52-70)
TVQSYTNFIRAVRGRLTT
>bi|19526601|geb|AAL87006.1| chain A [Viscum album] (74-92)
NLYVVAYQAGDQSYFLRD

Jean-Pierre.

hey! this is wht I ws exactly looking for. Thank you so much Jean.

you know what Jean? It didn't work initially. Then I removed the field separator - tab to keep the space as default value and I also changed it in FILE2. Then it worked perfectly.

Before also I noticed that ths field separator value does't work in my shell. Tell me if you know why this can happen.

Thanks again.
smriti

I run that script on files which need to extract longer substrings and the format of output file is getting disturbed. I tried some options of printf but coudn't correct it. Please help me out.

===FILE1====
>bi|37779709|geb|AAP20876.1| kinetic [ ternata]
MASKLLLFLLPAILGLIIPRPAVAVGTNYLLSGETLDTDGHLKNGDFDFIMQEDCNAVLYNGNWQSNTAN
KGRDCKLTLTDRGELVINNGEGSAVWRSGSQSAKGNYAAVLHPEGKLVIYGPSVFKINPWVPGLNSLRLG
NVPFTCNMLFSGQVLYGDGKITARNHMLVMQGDCNLVLYGGKCDWQSNTHGNGEHCFLRLNHKGELIIKD
DDFKSIWSSQSSSKQGDYVFILQDNGYGVIYGPAIWATSSKRSVAAQETMIGMVTEKVN
>bi|146403769|geb|ABQ32294.1| pure [an eg]
MAKLLLFLLPAILGLLIPRSAVALGTNYLLSGQTLNTDGHLKNGDFDLVMQNDCNLVLYNGNWQSNTANN
GRDCKLTLTDYGELVIKNGDGSTVWRSRAKSVKGNYAAVLHPDGRLVVFGPSVFKIDPWVPGLNSLRFRN
IPFTDNLLFSGQVLYGDGRLTAKNHQLVMQGDCNLVLYGGKYGWQSNTHGNGEHCFLRLNHKGELIIKDD
DFRPSGAAVPAPSR

===FILE2====
bi|37779709|geb|AAP20876.1| 28 264
bi|146403769|geb|ABQ32294.1| 27 224

===OUTPUTFILE===
>gi|37779709|gb|AAP20876.1| lectin [Pinellia ternata] (28-264)
NYLLSGETLDTDGHLKNGDFDFIMQEDCNAVLYNGNWQSNTANKGRDCKLTLTDRGELVINNGEGSAVWRSGSQSAKGNYAAVLHPEGKLVIYGPSVFKI NPWVPGLNSLRLGNVPFTCNMLFSGQVLYGDGKITARNHMLVMQGDCNLVLYGGKCDWQSNTHGNGEHCFLRLNHKGELIIKDDDFKSIWSSQSSSKQGD YVFILQDNGYGVIYGPAIWATSSKRSVAAQETMIGM
>gi|146403769|gb|ABQ32294.1| lectin [Colocasia esculenta] (27-224)
NYLLSGQTLNTDGHLKNGDFDLVMQNDCNLVLYNGNWQSNTANNGRDCKLTLTDYGELVIKNGDGSTVWRSRAKSVKGNYAAVLHPDGRLVVFGPSVFKI DPWVPGLNSLRFRNIPFTDNLLFSGQVLYGDGRLTAKNHQLVMQGDCNLVLYGGKYGWQSNTHGNGEHCFLRLNHKGELIIKDDDFRPSGAAVPAPS


where as the output should be like this:
>gi|37779709|gb|AAP20876.1| lectin [Pinellia ternata] (28-264)
NYLLSGETLDTDGHLKNGDFDFIMQEDCNAVLYNGNWQSNTANKGRDCKLTLTDRGELVINNGEGSAVWR
SGSQSAKGNYAAVLHPEGKLVIYGPSVFKINPWVPGLNSLRLGNVPFTCNMLFSGQVLYGDGKITARNHML
VMQGDCNLVLYGGKCDWQSNTHGNGEHCFLRLNHKGELIIKDDDFKSIWSSQSSSKQGDYVFILQDNGY
GVIYGPAIWATSSKRSVAAQETMIGM
>gi|146403769|gb|ABQ32294.1| lectin [Colocasia esculenta] (27-224)
NYLLSGQTLNTDGHLKNGDFDLVMQNDCNLVLYNGNWQSNTANNGRDCKLTLTDYGELVIKNGDGSTVWR
SRAKSVKGNYAAVLHPDGRLVVFGPSVFKIDPWVPGLNSLRFRNIPFTDNLLFSGQVLYGDGRLTAKNHQLV
MQGDCNLVLYGGKYGWQSNTHGNGEHCFLRLNHKGELIIKDDDFRPSGAAVPAPS

where each line after header line should not have more than 70 characters.

I will be thankful to you.
-smriti

Should the overlong lines be truncated or folder over multiple lines, or what?

To truncate (this truncates to eight characters; feel free to add more dots):

Code:

sed -e 's/^\(........\).*/\1/'

To fold, have a look at the fold command, or maybe use something like

Code:

sed -e 's/\(.......\)\(.\)/\1\
\2/g'

(Yes, that's a literal newline, to do the folding.) Again, this example folds at every eight characters; feel free to add more dots. (Actually the first line will be one character shorter. I'm too lazy to fix that. Or rather, the fix will depend on your sed dialect, and I don't want to go there.)

This could just as well be done with awk or perl or the truncation even with cut.

I don't want to truncate it and cut will also do that only. one more thing is that i want to apply ths folding only on lines after the line with '>' symbol.

and
sed -e 's/\(.......\)\(.\)/\1\
\2/g'
is giving a vague output ---
It is giving a / after every eight characters.

I gave the command as
sed -e 's/\(.......\)\(.\)/\1\\2/g' filename

Thanks
-smriti

No, there should be a literal newline between \1\ and \2

Anyway, fold should not touch lines which are shorter than 70 characters. As long as your > lines are shorter than that, you should be fine.

Here's another one for you:

Code:

perl -pe 's/(.{70})(?!$)/$1\n/g unless m/^>/'