Sponsored Content
Top Forums Shell Programming and Scripting Performance issue with awk script. Post 302213397 by RRVARMA on Thursday 10th of July 2008 02:40:22 AM
Old 07-10-2008
sample records.

Hi fpmurphy & radoulov,

Thanks for the feed back.

These are the sample records for first file file_3uniq_fields.out

Code:
1TVAO|OVEPT|VO
1TVAO|OVPDM|VO
6NFXE|17CLP|DH
6NFXE|NRZO4|EQ
6NFXE|SMOSA|EQ
ACA15|11X1W|DX
ACA15|1LN88|DX
ACA15|1LNSK|DX
ACA15|1LNVX|DX
ACA15|1LNVX|FD

and here are the sample records for second file.. file_sort_all_fields.out

Code:
1TVAO|S3zS033306|4577777770|4513201000|AJBFGJ|CB10|1TVAO|S3WS033306|4513101000|4513201000|AJBFGJ|CB10|OVEPT|VO|430300|430300|430300|009|IC    |Z|N|Y|IS
1TVAO|S3zS033306|4515685200|4513201000|AJBFGJ|CB10|1TVAO|S3WS033306|4513101000|4513201000|AJBFGJ|CB10|OVPDM|VO|430300|430300|430300|009|IC    |Z|N|Y|IS
6NFXE|S3Sr021401|4522451000|4511201000|B7BXHT|CB10|6NFXE|S3SN021401|4511101000|4511201000|B7BXHT|CB10|17CLP|DH|******|6670NI|410402|011|LQ    |Z|A|Y|IS
AGRJE|NA|NA|NA|NA|NA|6NFXE|S3SN021401|4511101000|4511201000|B7BXHT|CB10|NRZO4|EQ|402100|6670DC|410402|001|EQ|Z|U|Y|VT
6NFXE|S3Sz021401|4522201000|4511201000|B7BXHT|CB10|6NFXE|S3SN021401|4511101000|4511201000|B7BXHT|CB10|SMOSA|EQ|******|6670NI|410402|016|EQ    |Z|U|Y|IS
ACA15|S3Bz100120|4522201000|4511201000|AEBDHZ|CB10|ACA15|S3BW100120|4511101000|4511201000|AEBDHZ|CB10|11X1W|DX|410312|410312|410312|011|LQ    |Z|A|Y|IS
ACA15|S3BW100120|4512541000|4511201000|AEBDHZ|CB10|ACA15|S3BW100120|4511101000|4511201000|AEBDHZ|CB10|1LN88|DX|410312|410312|410312|A14|IOC   |Z|N|Y|IS
ARCXE|NA|NA|NA|NA|NA|ACA15|S3BW200120|4511101000|4511201000|AEBDHZ|CB10|1LN88|DX|410312|420100|420100|A14|IOC   |Z|N|Y|IS
ACA15|NA|NA|NA|NA|NA|ACA15|NA|NA|NA|NA|NA|1LNSK|DX|410312|410312|410312|A14|TC    |Z|N|Y|IS
ACA15|NA|NA|NA|NA|NA|ACA15|NA|NA|NA|NA|NA|1LNVX|DX|410312|410312|410312|009|IOC   |Z|N|Y|IS

Thanks,
RRVARMA
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

performance issue using gzcat, awk and sort

hi all, I was able to do a script to gather a few files and sort them. here it is: #!/usr/bin/ksh ls *mainFile* |cut -c20-21 | sort > temp set -A line_array i=0 file_name='temp' while read file_line do line_array=${file_line} let i=${i}+1 (5 Replies)
Discussion started by: naoseionome
5 Replies

2. Shell Programming and Scripting

Performance issue with ftp script.

Hi All, I have written a script to FTP files from local server to remote server. When i try it for few number of files the scripts runs successfully. But the same script when i run for 200-300 files it gives me performanace issue by aborting the connection. Please help me out to improve the... (7 Replies)
Discussion started by: Shiv@jad
7 Replies

3. UNIX for Advanced & Expert Users

FTP-Shell Script-Performance issue

Hello All, Request any one of Unix/Linux masters to clarify on the below. How far it is feasible to open a new ftp connection for transferring each file when there are multiple files to be sent. I have developed shell script to send all files at single stretch but some how it doesnt suit to... (3 Replies)
Discussion started by: RSC1985
3 Replies

4. Solaris

Performance issue

Hi Gurus, I am beginner in solaris and want to know what are the things we need to check for performance monitoring on our solairs OS. for DISK,CPU and MEMORY. Also how we do ipforwarding in slaris Many thanks for your help Pradeep P (4 Replies)
Discussion started by: ppandey21
4 Replies

5. Shell Programming and Scripting

Script performance issue

hi i have written a shell script which comapare a text file data with files within number of different directories. example. Text File: i have a file /u02/abc.txt which have almost 20000 file names Directories: i have a path /u03 which have some subdirectories like a,b,c which have almost... (2 Replies)
Discussion started by: malikshahid85
2 Replies

6. UNIX for Dummies Questions & Answers

Performance issue

hi I am having a performance issue with the following requirement i have to create a permutation and combination on a set of three files such that each record in each file is picked and the output is redirected in a specific format but it is taking around 70 odd hours to prepare a combination... (7 Replies)
Discussion started by: mad_man12
7 Replies

7. Shell Programming and Scripting

awk performance issue

Hi, I have the code below as cat <filename> | tr '~' '\n' | sed '/^$/ d' | sed "s/*/|/g" > <filename> awk -F\| -vt=`date +%m%d%y%H%M%S%s` '$1=="ST",$1=="SE"{if($1=="ST"){close(f);f="214_edifile_"t"" ++i} ; $1=$1; print>f}' OFS=\| <filename> This script replaces some characters and... (4 Replies)
Discussion started by: atlantis_yy
4 Replies

8. UNIX for Dummies Questions & Answers

awk script performance issue

Hello All, I have the below excerpt of code in my shell script and it taking long time to complete, though it prints the output quickly. Is there a way to make it come out once it finds the first instance as the file size of 4.7 GB it could be going through all lines of the data file to find for... (3 Replies)
Discussion started by: Ariean
3 Replies

9. Shell Programming and Scripting

Performance issue in shell script

Hi All, I am facing performance issue while rinning the LINUX shell script. I have file1 and file 2. File one is the source file and file 2 is lookup file. Need to replace if the pattern is matching in file1 with file2. The order of lookup file is important as if any match then exit... (8 Replies)
Discussion started by: ureddy
8 Replies

10. Shell Programming and Scripting

Performance Issue - Shell Script

Hi, I am beginner in shell scripting. I have written a script to parse file(s) having large number of lines each having multiple comma separated strings. But it seems like script is very slow. It took more than 30mins to parse a file with size 120MB (523564 lines), below is the script code ... (4 Replies)
Discussion started by: imrandec85
4 Replies
Bio::Tools::Run::EMBOSSacd(3pm) 			User Contributed Perl Documentation			   Bio::Tools::Run::EMBOSSacd(3pm)

NAME
Bio::Tools::Run::EMBOSSacd - class for EMBOSS Application qualifiers SYNOPSIS
use Bio::Factory::EMBOSS; # get an EMBOSS application object from the EMBOSS factory $factory = Bio::Factory::EMBOSS->new(); $application = $factory->program('embossversion'); # run the application with an optional hash containing parameters $result = $application->run(); # returns a string or creates a file print $result . " "; $water = $factory->program('water'); # here is an example of running the application # water can compare 1 seq against 1->many sequences # in a database using Smith-Waterman my $seq_to_test; # this would have a seq here my @seqs_to_check; # this would be a list of seqs to compare # (could be just 1) my $wateroutfile = 'out.water'; $water->run({ -sequencea => $seq_to_test, -seqall => @seqs_to_check, -gapopen => '10.0', -gapextend => '0.5', -outfile => $wateroutfile}); # now you might want to get the alignment use Bio::AlignIO; my $alnin = Bio::AlignIO->new(-format => 'emboss', -file => $wateroutfile); while( my $aln = $alnin->next_aln ) { # process the alignment -- these will be Bio::SimpleAlign objects } DESCRIPTION
The EMBOSSacd represents all the possible command line arguments that can be given to an EMBOSS application. Do not create this object directly. It will be created and attached to its corresponding Bio::Tools::Run::EMBOSSApplication if you set $application->verbose > 0 Call $application->acd to retrive the Bio::Tools::Run::EMBOSSApplication::EMBOSSacd object. See also Bio::Tools::Run::EMBOSSApplication and Bio::Factory::EMBOSS. FEEDBACK
Mailing Lists User feedback is an integral part of the evolution of this and other Bioperl modules. Send your comments and suggestions preferably to the Bioperl mailing lists Your participation is much appreciated. bioperl-l@bioperl.org - General discussion http://bioperl.org/wiki/Mailing_lists - About the mailing lists Support Please direct usage questions or support issues to the mailing list: bioperl-l@bioperl.org rather than to the module maintainer directly. Many experienced and reponsive experts will be able look at the problem and quickly address it. Please include a thorough description of the problem with code and data examples if at all possible. Reporting Bugs report bugs to the Bioperl bug tracking system to help us keep track the bugs and their resolution. Bug reports can be submitted via the web: http://redmine.open-bio.org/projects/bioperl/ AUTHOR - Heikki Lehvaslaiho Email: heikki-at-bioperl-dot-org Address: EMBL Outstation, European Bioinformatics Institute Wellcome Trust Genome Campus, Hinxton Cambs. CB10 1SD, United Kingdom APPENDIX
The rest of the documentation details each of the object methods. Internal methods are usually preceded with a _ new Title : new Usage : $emboss_prog->acd($prog_name); Function: Constructor for the class. Calls EMBOSS program 'acdc', converts the HTML output into XML and uses XML::Twig XML parser to write out a hash of qualifiers which is then blessed. Throws : without program name Returns : new object Args : EMBOSS program name name Title : name Usage : $embossacd->name Function: sets/gets the name of the EMBOSS program Setting is done by the EMBOSSApplication object, you should only get it. Throws : Returns : name string Args : None print Title : print Usage : $embossacd->print; $embossacd->print('-word'); Function: Print out the qualifiers. Uses Data::Dumper to print the qualifiers into STDOUT. A valid qualifier name given as an argment limits the output. Throws : Returns : print string Args : optional qualifier name mandatory Title : mandatory Usage : $acd->mandatory Function: gets a mandatory subset of qualifiers Throws : Returns : Bio::Tools::Run::EMBOSSacd object Args : none Qualifier queries These methods can be used test qualifier names and read values. qualifier Title : qualifier Usage : $acd->qualifier($string) Function: tests for the existence of the qualifier Throws : Returns : boolean Args : string, name of the qualifier category Title : category Usage : $acd->category($qual_name) Function: Return the category of the qualifier Throws : Returns : 'mandatory' or 'optional' or 'advanced' or 'associated' or 'general' Args : string, name of the qualifier values Title : values Usage : $acd->values($qual_name) Function: Return the possible values for the qualifier Throws : Returns : string Args : string, name of the qualifier descr Title : descr Usage : $acd->descr($qual_name) Function: Return the description of the qualifier Throws : Returns : boolean Args : string, name of the qualifier unnamed Title : unnamed Usage : $acd->unnamed($qual_name) Function: Find if the qualifier can be left unnamed Throws : Returns : 0 if needs to be named, order number otherwise Args : string, name of the qualifier default Title : default Usage : $acd->default($qual_name) Function: Return the default value for the qualifier Throws : Returns : scalar Args : string, name of the qualifier perl v5.12.3 2011-06-18 Bio::Tools::Run::EMBOSSacd(3pm)
All times are GMT -4. The time now is 05:38 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy