How to extract some parts of a file to create some outfile

iammitra · #1 (**permalink**) 05-08-2009

Hi All,
I am very new in programming. I need some help.
I have one input file like:

Number of disabled taxa: 9
Loading mapping file: ncbi.map
Load mapping:
taxId2TaxLevel: 469951
--- Subsample reads (20%): 66680 of 334386
Processing: tree-from-summary
Running tree-from-summary algorithm
Taxonomy:
Gammaproteobacteria: 2767
Alphaproteobacteria: 4123
Deltaproteobacteria: 1343
Epsilonproteobacteria: 26
Not assigned: 1445
No hits: 220253
+++++++++++End of summary for file: B-Red-sum.txt
--- Subsample reads (20%): 67037 of 334386
Processing: tree-from-summary
Running tree-from-summary algorithm
Taxonomy:
Gammaproteobacteria: 2809
Alphaproteobacteria: 4001
Deltaproteobacteria: 1208
Epsilonproteobacteria: 15
Not assigned: 299
No hits: 461890
+++++++++++End of summary for file: B-Red-sum.txt

::::: and so on

I want to create some output like:
Out file1.txt(which grep from, next line of "Taxonomy:" upto "+++++++++++End" ) with no space in front of line and so on.

So the desired ouput will be:
outfile1.txt
Gammaproteobacteria: 2767
Alphaproteobacteria: 4123
Deltaproteobacteria: 1343
Epsilonproteobacteria: 26
Not assigned: 1445
No hits: 220253

outfile2.txt
Gammaproteobacteria: 2809
Alphaproteobacteria: 4001
Deltaproteobacteria: 1208
Epsilonproteobacteria: 15
Not assigned: 299
No hits: 461890

and so on.

Can anybody please help me in this matter?

I tried with some code like this. But didn't workout.
--------------------------------------------------------------------------
#!/bin/tcsh
if $#argv != "1" then
echo "Usage: process-file-script 1st-output-file-as-inputfile"
exit 0
endif

FIL_NM=$1

str=""
cat $FIL_NM | while read LINE
do
if [ "`echo $LINE | awk '{print $1}'`" = "+++++++++++Begin" ] ; then
n=1
c=1
fi
if [ "`echo $LINE |grep Gamma`"] ; then
NEW_FIL_NM=$FIL_NM"_"$n.txt"
fi

fi
if [ "`echo $LINE | awk '{print $1}'`" = "+++++++++++End" ] ; then
n=0
fi
done
--------------------------------------------------------
Please help...
Many thanks in advance...
Best wishes,
Mitra

vgersh99 · #2 (**permalink**) 05-08-2009

Code:

nawk '
    /^Taxonomy/ {p=6;close(out);out="output" ++cnt ".txt";next}
    p &&p-- { print > out }' myInputFile

ghostdog74 · #3 (**permalink**) 05-08-2009

if you have Python, here's an alternative solution

Code:

f=0;i=0
for line in open("file"):
    line=line.strip()
    if line.startswith("+++++++++++"): 
        f=0
        o.close()
    if "Taxonomy:" in line: 
        f=1;i=i+1
        o=open("out_"+str(i)+".txt","w")
    if f:
        print >>o, line

durden_tyler · #4 (**permalink**) 05-08-2009

And here's a perl solution:

Code:

$
$
$ cat input.txt
Number of disabled taxa: 9
Loading mapping file: ncbi.map
Load mapping:
taxId2TaxLevel: 469951
--- Subsample reads (20%): 66680 of 334386
Processing: tree-from-summary
Running tree-from-summary algorithm
Taxonomy:
Gammaproteobacteria: 2767
Alphaproteobacteria: 4123
Deltaproteobacteria: 1343
Epsilonproteobacteria: 26
Not assigned: 1445
No hits: 220253
+++++++++++End of summary for file: B-Red-sum.txt
--- Subsample reads (20%): 67037 of 334386
Processing: tree-from-summary
Running tree-from-summary algorithm
Taxonomy:
Gammaproteobacteria: 2809
Alphaproteobacteria: 4001
Deltaproteobacteria: 1208
Epsilonproteobacteria: 15
Not assigned: 299
No hits: 461890
+++++++++++End of summary for file: B-Red-sum.txt
::::: and so on
$
$
$
$ perl -ne '{$/=""; $i=1;
>   while (/^Taxonomy:.(.*?)\+{11}/msgi) {
>     open(OUT,">outfile".$i++.".txt"); print OUT $1; close(OUT);
>   }}' input.txt
$
$
$ cat outfile1.txt
Gammaproteobacteria: 2767
Alphaproteobacteria: 4123
Deltaproteobacteria: 1343
Epsilonproteobacteria: 26
Not assigned: 1445
No hits: 220253
$
$
$ cat outfile2.txt
Gammaproteobacteria: 2809
Alphaproteobacteria: 4001
Deltaproteobacteria: 1208
Epsilonproteobacteria: 15
Not assigned: 299
No hits: 461890
$
$

tyler_durden

iammitra · #5 (**permalink**) 05-08-2009

Hallo ghostdog74,
Thanks for your reply. But I am sorry to say that I forgot to mention : in my input file there are not always only 6 lines. I just copied some lines.. This lines varies from 100 to 200. So it is necessary for the program to read +++++++++End.

Thanks a lot,
Mitra.

ghostdog74 · #6 (**permalink**) 05-08-2009

Quote:

Originally Posted by iammitra

Hallo ghostdog74,
Thanks for your reply. But I am sorry to say that I forgot to mention : in my input file there are not always only 6 lines. I just copied some lines.. This lines varies from 100 to 200. So it is necessary for the program to read +++++++++End.

Thanks a lot,
Mitra.

well, i am not sure i get you, but i see other solutions include "End', therefore if you are sure that ++++++++ is not unique, you can add "End"

Code:

....
if line.startswith("+++++++++++End"): 
....

summer_cherry · #7 (**permalink**) 05-11-2009

below perl code should help you some.

Code:

open $fh,"<","a.txt";
my ($flag,$n)=(0,0);
while(<$fh>){
	if(/Taxonomy:/){
		$n++;
		$file=sprintf("outfile%s.txt",$n);
		open FH,"+>$file";
		$flag=1;
		next;
	}
	if(/\++/){
		$flag=0;
		next;
	}
	print FH $_ if $flag==1;
}

More UNIX and Linux Forum Topics You Might Find Helpful
Thread	Thread Starter	Forum	Replies	Last Post
extract columns from 2 different files and create new file	Amit.Sagpariya	Shell Programming and Scripting	1	04-30-2009 07:23 AM
Setting outfile with (brackets)	Pablo_beezo	Shell Programming and Scripting	1	10-02-2008 09:15 AM
getting parts of a file	bebop1111116	Shell Programming and Scripting	11	10-09-2006 10:19 AM
filter parts of a big file using awk or sed script	apalex	Shell Programming and Scripting	1	07-25-2005 04:45 PM
cksum parts of a file	crazykelso	UNIX for Dummies Questions & Answers	6	07-30-2002 11:38 AM

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes	Rate This Thread
Switch to Linear Mode Hybrid Mode Switch to Threaded Mode	Rate This Thread: