I was running around of 200 djob for a Blastp search in a cluster. All my input files were protein fasta file (prot.fna.1, prot.fna.2 ...prot.fna.200). The output of each individual slurm job is located in a corresponding file ending with *test (prot.fna.1.test, prot.fna.2.test ...prot.fna.200.test) in the same directory. Unfortunately, these Jobs were canceled due to time limit on the node. Now I want to extract all the remaining sequences from my protein fasta files a way to run them again and all the results could be concatenated. Here his what I doing :
1. I look for the first string of one *test file with this command:
Code:
(awk '{print $1}' prot.fna.1.test | tail -n1)
, this scrip print me the “pattern”
2. All the sequences after this matching pattern in the corresponding fasta input (prot.fasta.1) is printed using this command :
Code:
cat prot.fasta.1 | sed -e '1,/pattern/ d' | sed -ne '/^>/,$ p'
Repeating this for 200 files one by one is time consuming. I want to run this script in all the files , but I can't. I am writing you to see if you can help me implement this please. Here is what I am doing using these scripts :
Code:
[dderilus@boqueron ERR598955_orfm_250_out]$ awk '{print $1}' ERR598955_orfm.fna.11.test | tail -n1
ERR598955.6981687_74_5_4
[dderilus@boqueron ERR598955_orfm_250_out]$ cat ERR598955_orfm.fna.11 | sed -e '1,/ERR598955.6981687_74_5_4/ d' | sed -ne '/^>/,$ p' > first_out.fna1.11
[dderilus@boqueron ERR598955_orfm_250_out]$ awk '{print $1}' ERR598955_orfm.fna.12.test | tail -n1
ERR598955.7664144_89_2_3
[dderilus@boqueron ERR598955_orfm_250_out]$ cat ERR598955_orfm.fna.1 | sed -e '1,/ERR598955.7664144_89_2_3/ d' | sed -ne '/^>/,$ p' > first_out.fna1.12
[dderilus@boqueron ERR598955_orfm_250_out]$ less first_out.fna1.12
[dderilus@boqueron ERR598955_orfm_250_out]$ cat ERR598955_orfm.fna.12 | sed -e '1,/ERR598955.7664144_89_2_3/ d' | sed -ne '/^>/,$ p' > first_out.fna1.12
[dderilus@boqueron ERR598955_orfm_250_out]$ awk '{print $1}' ERR598955_orfm.fna.13.test | tail -n1
ERR598955.8364684_101_2_4
[dderilus@boqueron ERR598955_orfm_250_out]$ cat ERR598955_orfm.fna.13 | sed -e '1,/ERR598955.8364684_101_2_4/ d' | sed -ne '/^>/,$ p' > first_out.fna1.13
[dderilus@boqueron ERR598955_orfm_250_out]$ awk '{print $1}' ERR598955_orfm.fna.14.test | tail -n1
ERR598955.9053411_57_6_5
[dderilus@boqueron ERR598955_orfm_250_out]$ cat ERR598955_orfm.fna.14 | sed -e '1,/ERR598955.9053411_57_6_5/ d' | sed -ne '/^>/,$ p' > first_out.fna1.14
[dderilus@boqueron ERR598955_orfm_250_out]$ awk '{print $1}' ERR598955_orfm.fna.15.test | tail -n1
ERR598955.9746341_78_3_2
[dderilus@boqueron ERR598955_orfm_250_out]$ cat ERR598955_orfm.fna.15 | sed -e '1,/ERR598955.9746341_78_3_2/ d' | sed -ne '/^>/,$ p' > first_out.fna1.15
[dderilus@boqueron ERR598955_orfm_250_out]$ awk '{print $1}' ERR598955_orfm.fna.16.test | tail -n1
ERR598955.10426164_9_3_3
[dderilus@boqueron ERR598955_orfm_250_out]$ cat ERR598955_orfm.fna.16 | sed -e '1,/ERR598955.10426164_9_3_3/ d' | sed -ne '/^>/,$ p' > first_out.fna1.16
[dderilus@boqueron ERR598955_orfm_250_out]$ awk '{print $1}' ERR598955_orfm.fna.17.test | tail -n1
ERR598955.11123991_2_2_2
[dderilus@boqueron ERR598955_orfm_250_out]$ cat ERR598955_orfm.fna.17 | sed -e '1,/ERR598955.11123991_2_2_2/ d' | sed -ne '/^>/,$ p' > first_out.fna1.17
[dderilus@boqueron ERR598955_orfm_250_out]$ awk '{print $1}' ERR598955_orfm.fna.18.test | tail -n1
ERR598955.11810206_3_6_1
[dderilus@boqueron ERR598955_orfm_250_out]$ cat ERR598955_orfm.fna.18 | sed -e '1,/ERR598955.11810206_3_6_1/ d' | sed -ne '/^>/,$ p' > first_out.fna1.18
[dderilus@boqueron ERR598955_orfm_250_out]$ awk '{print $1}' ERR598955_orfm.fna.19.test | tail -n1
ERR598955.12519405_1_4_4
[dderilus@boqueron ERR598955_orfm_250_out]$ cat ERR598955_orfm.fna.19 | sed -e '1,/ERR598955.12519405_1_4_4/ d' | sed -ne '/^>/,$ p' > first_out.fna1.19
This is time consuming, I would be very grateful if you can help me to do that with one script.
Thanks in advance
Cordially
Moderator's Comments:
Please add code tags!
Last edited by MadeInGermany; 06-04-2018 at 05:22 AM..
Reason: Added code tags, removed font tags
Dear folks
I have two data set which there names are "final.map" and "1.geno" and look like this structures:
final.map:
gi|358485511|ref|NC_006088.3| 2044
gi|358485511|ref|NC_006088.3| 2048
gi|358485511|ref|NC_006088.3| 2187
gi|358485511|ref|NC_006088.3| 17654
... (2 Replies)
How can I Run one script on multiple files and print out multiple files.
FOR EXAMPLE
i want to run script.pl on 100 files named 1.txt ....100.txt under same directory and print out corresponding file 1.gff ....100.gff.THANKS (4 Replies)
How can I run the following command on multiple files and print out the corresponding multiple files.
perl script.pl genome.gff 1.txt > 1.gff
However, there are multiples files of 1.txt, from 1----100.txt
Thank you so much.
No duplicate posting! Continue here. (0 Replies)
Hi everyone,
I'm new to the forums, as you can probably tell... I'm also pretty new to scripting and writing any type of code.
I needed to know exactly how I can grep for multiple strings, in files located in one directory, but I need each string to output to a separate file.
So I'd... (19 Replies)
hi,
I have a directory "test" under which there are 3 files a.txt,b.txt and c.txt.
I need to rename those files to a.pl,b.pl and c.pl respectively.
is it possible to achieve this in a sed or awk one liner?
i have searched but many of them are scripts.
I need to do this in a one liner.
I... (2 Replies)
Hi,
I'd like to process multiple files. For example:
file1.txt
file2.txt
file3.txt
Each file contains several lines of data. I want to extract a piece of data and output it to a new file.
file1.txt ----> newfile1.txt
file2.txt ----> newfile2.txt
file3.txt ----> newfile3.txt
Here is... (3 Replies)
Hi guys,
say I have a few files in a directory (58 text files or somthing)
each one contains mulitple strings that I wish to replace with other strings
so in these 58 files I'm looking for say the following strings:
JAM (replace with BUTTER)
BREAD (replace with CRACKER)
SCOOP (replace... (19 Replies)
Hi,
I need help to split lines from a file into multiple files.
my input look like this:
13
23 45 45 6 7
33 44 55 66 7
13
34 5 6 7 87
45 7 8 8 9
13
44 55 66 77 8
44 66 88 99 6
I want to split every 3 lines from this file to be written to individual files. (3 Replies)
I'm trying some thing like this. But not working
It worked for bash files
Now I want some thing like that along with multiple input files by redirecting their outputs as inputs of next command like below
Could you guyz p0lz help me on this
#!/usr/bin/awk -f
BEGIN
{
}
script1a.awk... (2 Replies)
Hello
when I try to run rm on multiple files I have problem to delete files with space.
I have this command :
find . -name "*.cmd" | xargs \rm -f
it doing the work fine but when it comes across files with spaces like : "my foo file.cmd"
it refuse to delete it
why? (1 Reply)
06-04-2018
