Sponsored Content
Top Forums Shell Programming and Scripting using awk removing newline and specific position Post 302405437 by durden_tyler on Thursday 18th of March 2010 10:48:39 PM
Old 03-18-2010
Here's a Perl solution -

Code:
$ 
$ cat -n f8
     1  >FASTA Header1
     2  line1
     3  line2
     4  line3
     5  linen
     6  >FASTA Header2
     7  Line1
     8  Line2
     9  linen
    10  >FASTA Header3
    11  Line1
    12  Line2
    13  linen
$ 
$ perl -lne 'if (/^>/){print "\n",$_} else {printf} END{print}' f8

>FASTA Header1
line1line2line3linen
>FASTA Header2
Line1Line2linen
>FASTA Header3
Line1Line2linen
$

There are no arrays involved here. I tested it on a dummy file that has 250,000 "FASTA Headers" and 4 lines under each FASTA header. So that's 1,250,000 lines in all.

Code:
$ 
$ head f88
>FASTA Header1
line1
line2
line3
line4
>FASTA Header2
line1
line2
line3
line4
$ 
$ tail f88
>FASTA Header249999
line1
line2
line3
line4
>FASTA Header250000
line1
line2
line3
line4
$ 
$ wc f88
 1250000  1500000 10888895 f88
$ 
$

The Perl script, when run on this file, and redirected to another file, takes roughly 5 seconds on my system.

Code:
$ 
$ 
$ time perl -lne 'if (/^>/){print "\n",$_} else {printf} END{print}' f88 >f89

real    0m5.454s
user    0m3.456s
sys     0m0.179s
$ 
$

HTH,
tyler_durden
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sed removing carriage return and newline

Hi, I'm not very familiar with unix shell. I want to replace the combination of two carriage returns and one newline with one carriage return and one newline. I think the best way to do this is to use sed. I tried something like this: sed -e "s#\#\#g" file.txt but it doesn't work. Thanx... (2 Replies)
Discussion started by: mored
2 Replies

2. Shell Programming and Scripting

Using sed to replace specific character and specific position

I am trying to use sed to replace specific characters at a specific position in the file with a different value... can this be done? Example: File: A0199999123 A0199999124 A0199999125 Need to replace 99999 in positions 3-7 with 88888. Any help is appreciated. (5 Replies)
Discussion started by: programmer22
5 Replies

3. Shell Programming and Scripting

AWK or SED to add string at specific position

Greetings. I don't have experience programing scripts. I need to insert a string in a specific position of another string on another file (last.cfg), for example: File last.cfg before using script: login_interval=1800 lcs.machinename=client04 File last.cfg after using script:... (4 Replies)
Discussion started by: vanesuke
4 Replies

4. Shell Programming and Scripting

Remove text from n position to n position sed/awk

I want to remove text from nth position to nth position couple of times in same line my line is "hello is there anyone can help me with this question" I need like this ello is there anyone can help me with question 'h' is removed and 'this' removed from the line. I want to do this... (5 Replies)
Discussion started by: elamurugu
5 Replies

5. Shell Programming and Scripting

substitute a string on a specific position for specific lines

I woud like to substitue a string on a specific position for specific lines I've got a file and I would like to change a specific string from "TOCHANGE" to "ABCABCAB" For every line (except 1,2, 3 and the last one) , I need to check between the 9th and the 16th digits. For the 3rd line, I... (7 Replies)
Discussion started by: BSF
7 Replies

6. Shell Programming and Scripting

awk removing specific line

Hi, I have file with lines key=val. For ex(conf.file):- c=3 ef=78 b=40 ca=40 I want to remove the line with c=3, when I execute the below command it's removing both lines i.e c=3 and ca=40. Could you please correct my command. /usr/xpg4/bin/awk -v key=c 'match($0,key)... (3 Replies)
Discussion started by: axes
3 Replies

7. Shell Programming and Scripting

Removing 0 from a specific position - if it exists

I have a file that I need to parse using a script. The dates in the file are displayed in the format: Mar 2, 2011 9:09:31 PM I have tried using the date command %e and %l but it pads an extra space for the day and hour if they are single digits. So this I used a normal date command: ... (6 Replies)
Discussion started by: crazyideas
6 Replies

8. Shell Programming and Scripting

Removing ^M and the newline that follows it.

Hi Gurus, Apologies as I feel like this must be answered already on here somewhere but I just can't find it. I find many people looking to remove all \n and \r (CR and LF) or one or the other but the only times I've found someone trying to remove them only when both are together they've found... (7 Replies)
Discussion started by: Leedor
7 Replies

9. Shell Programming and Scripting

Awk command to replace specific position characters.

Hi, I have a fixed width file. The way this file works is say for example there are 30 columns in it each with different sizes say 10,5,2, etc... If data in a field is less than the field size the rest of it is loaded with spaces. I would like an awk command to that would replace I have... (8 Replies)
Discussion started by: pinnacle
8 Replies

10. Linux

Removing a character at specific position in a column

Hi, I have a file like this (about 8 columns in total, this being the 2nd column) gi_49482297_ref_YP_039521.1_ gi_49482297_ref_YP_039521.1_ gi_49482315_ref_YP_039539.1_ gi_49482315_ref_YP_039539.1_I want to remove the _ at the end of the line. And at later stages I would want to replace the... (5 Replies)
Discussion started by: Syeda Sumayya
5 Replies
FORMATDB(1)						     NCBI Tools User's Manual						       FORMATDB(1)

NAME
formatdb - format protein or nucleotide databases for BLAST SYNOPSIS
formatdb [-] [-B filename] [-F filename] [-L filename] [-T filename] [-V] [-a] [-b] [-e] [-i filename] [-l filename] [-n str] [-o] [-p F] [-s] [-t str] [-v N] DESCRIPTION
formatdb must be used in order to format protein or nucleotide source databases before these databases can be searched by blastall, blastpgp or MegaBLAST. The source database may be in either FASTA or ASN.1 format. Although the FASTA format is most often used as input to formatdb, the use of ASN.1 is advantageous for those who are using ASN.1 as the common source for other formats such as the GenBank report. Once a source database file has been formatted by formatdb it is not needed by BLAST. Please note that if you are going to apply periodic updates to your BLAST databases using fmerge(1), you will need to keep the source database file. OPTIONS
A summary of options is included below. - Print usage message -B filename Binary Gifile produced from the Gifile specified by -F. This option specifies the name of a binary GI list file. This option should be used with the -F option. A text GI list may be specified with the -F option and the -B option will produce that GI list in binary format. The binary file is smaller and BLAST does not need to convert it, so it can be read faster. -F filename Gifile (file containing list of gi's) for use with -B or -L -L filename Create an alias file named filename, limiting the sequences searched to those specified by -F. -T filename Set the taxonomy IDs in ASN.1 deflines according to the table in filename. -V Verbose: check for non-unique string ids in the database -a Input file is database in ASN.1 format (otherwise FASTA is expected) -b ASN.1 database is binary (as opposed to ASCII text) -e Input is a Seq-entry. A source ASN.1 database (either text ascii or binary) may contain a Bioseq-set or just one Bioseq. In the latter case -e should be provided. -i filename Input file(s) for formatting -l filename Log file name (default = formatdb.log) -n str Base name for BLAST files (defaults to the name of the original FASTA file) -o Parse SeqID and create indexes. If the source database is in FASTA format, the database identifiers in the FASTA definition line must follow the conventions of the FASTA Defline Format. -p F Input is a nucleotide, not a protein. -s Index only by accession, not by locus. This is especially useful for sequence sets like the EST's where the accession and locus names are identical. Formatdb runs faster and produces smaller temporary files if this option is used. It is strongly recommended for EST's, STS's, GSS's, and HTGS's. -t str Title for database file [String] -v N Break up large FASTA files into `volumes' of size N million letters (4000 by default). As part of the creation of a volume, for- matdb writes a new type of BLAST database file, called an alias file, with the extension `nal' or `pal'. AUTHOR
The National Center for Biotechnology Information. SEE ALSO
blast(1), copymat(1), formatrpsdb(1), makemat(1), /usr/share/doc/blast2/formatdb.html. NCBI
2007-10-19 FORMATDB(1)
All times are GMT -4. The time now is 09:23 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy