Sponsored Content
Special Forums UNIX and Linux Applications Infrastructure Monitoring Possible performance improvement (Bash and flat file) Post 302419494 by jim mcnamara on Friday 7th of May 2010 10:35:16 AM
Old 05-07-2010
One major thing to look at: child process creation. Try to use shell builtins instead of
a lot of back tic ` ` (or $(... ) ) constructions.

You can also store your flat file in a variable, so you read it only once:
Code:
flatfile=$(< /path/to/my/flatfile)

Then you can step thru the records or create arrays of the data.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Help with Flat Files Please!! BASH (New User)

Hello All, I am brand new to the UNIX world and so far and very intrigued and enjoy scripting. This is just a new language for me. I would really like assistance with the below request. Any help would be greatly appreciated! I want to create a flat file in Vi that has a header field and... (0 Replies)
Discussion started by: cyberjax21
0 Replies

2. Programming

File - reading - Performance improvement

Hi All I am reading a huge file of size 2GB atleast. I am reading each line and cutting certain columns and writing it to another file. Here is the logic. int main() { string u_line; string Char_List; string u_file; int line_pos; string temp_form_u_file; ... (10 Replies)
Discussion started by: dhanamurthy
10 Replies

3. Shell Programming and Scripting

Any improvement possible in this script

Hi! Thank you for the help yesterday This is the finished product There is one more thing I would like to do to it but I’m not to certain On how to proceed I would like to log all output to a log in order to Be able to roll back This script is meant to be used in repairing a... (4 Replies)
Discussion started by: Ex-Capsa
4 Replies

4. UNIX for Advanced & Expert Users

linux os improvement

can anyone help to share the knowledge on linux os improvement? 1) os account - use window AD authentication, such as ldap, but how to set /etc/passwd, where to put user home? 2) user account activity - how to log os user activity share the idea and what tools can do that...thx (5 Replies)
Discussion started by: goodbid
5 Replies

5. Shell Programming and Scripting

Display-performance in terminal, bash or python?

Heyas I've been working on my project TUI (Text User Interface) for quite some time now, its a hobby project, so nothing i sit in front of 8hrs/day. Since the only 'real' programming language i knw is Visual Basic, based upon early steps with MS-Batch files. When i 'joined' linux 3 years ago,... (7 Replies)
Discussion started by: sea
7 Replies

6. Shell Programming and Scripting

[BASH] Performance question - Script to STDOUT

Hello Coders Some time ago i was asking about python and bash performances, and i was told i could post the regarding code, and someone would kindly help to make it faster (if possible). If you have noted, i'm on the way to finalize, finish, stable TUI - Text(ual) User Interface. It is a... (6 Replies)
Discussion started by: sea
6 Replies

7. Shell Programming and Scripting

Performance improvement in grep

Below script is used to search numeric data from around 400 files in a folder. I have 300 such folders. Need help in performance improvement in the script. Below Script searches 20 such folders ( 300 files in each folder) simultaneously. This increases cpu utilization upto 90% What changes... (3 Replies)
Discussion started by: vegasluxor
3 Replies

8. Shell Programming and Scripting

Bash - array loop performance

Hi, another little question... "sn" is an array whose elements can vary from about 55,000 to about 150,000 elements. Each element consists of an integer between 0-255, eg: ${sn} contain the value: 103 . For a decrypt-procedure I need scroll all the elements 4 or 5 times. Here is an example of... (15 Replies)
Discussion started by: math4
15 Replies

9. OS X (Apple)

Create a bash array from a flat file of whitespaces only.

Hi guys and gals... MacBook Pro. OSX 10.13.2, default bash terminal. I have a flat file 1920 bytes in size of whitespaces only. I need to put every single whitespace character into a bash array cell. Below are two methods that work, but both are seriously ugly. The first one requires that I... (7 Replies)
Discussion started by: wisecracker
7 Replies

10. Shell Programming and Scripting

Bash script search, improve performance with large files

Hello, For several of our scripts we are using awk to search patterns in files with data from other files. This works almost perfectly except that it takes ages to run on larger files. I am wondering if there is a way to speed up this process or have something else that is quicker with the... (15 Replies)
Discussion started by: SDohmen
15 Replies
Boulder::Omim(3pm)					User Contributed Perl Documentation					Boulder::Omim(3pm)

NAME
Boulder::Omim - Fetch Omim data records as parsed Boulder Stones SYNOPSIS
# parse a file of Omim records $om = new Boulder::Omim(-accessor=>'File', -param => '/data/omim/omim.txt'); while (my $s = $om->get) { print $s->Identifier; print $s->Text; } # parse flatfile records yourself open (OM,"/data/omim/omim.txt"); local $/ = "*RECORD*"; while (<OM>) { my $s = Boulder::Omim->parse($_); # etc. } DESCRIPTION
Boulder::Omim provides retrieval and parsing services for OMIM records Boulder::Omim provides retrieval and parsing services for NCBI Omim records. It returns Omim entries in Stone format, allowing easy access to the various fields and values. Boulder::Omim is a descendent of Boulder::Stream, and provides a stream-like interface to a series of Stone objects. Access to Omim is provided by one accessors, which give access to local Omim database. When you create a new Boulder::Omim stream, you provide the accessors, along with accessor-specific parameters that control what entries to fetch. The accessors is: File This provides access to local Omim entries by reading from a flat file (typically omim.txt file downloadable from NCBI's Ftp site). The stream will return a Stone corresponding to each of the entries in the file, starting from the top of the file and working downward. The parameter is the path to the local file. It is also possible to parse a single Omim entry from a text string stored in a scalar variable, returning a Stone object. Boulder::Omim methods This section lists the public methods that the Boulder::Omim class makes available. new() # Local fetch via File $om=new Boulder::Omim(-accessor => 'File', -param => '/data/omim/omim.txt'); The new() method creates a new Boulder::Omim stream on the accessor provided. The only possible accessors is File. If successful, the method returns the stream object. Otherwise it returns undef. new() takes the following arguments: -accessor Name of the accessor to use -param Parameters to pass to the accessor Specify the accessor to use with the -accessor argument. If not specified, it defaults to File. -param is an accessor-specific argument. The possibilities is: For File, the -param argument must point to a string-valued scalar, which will be interpreted as the path to the file to read Omim entries from. get() The get() method is inherited from Boulder::Stream, and simply returns the next parsed Omim Stone, or undef if there is nothing more to fetch. It has the same semantics as the parent class, including the ability to restrict access to certain top-level tags. put() The put() method is inherited from the parent Boulder::Stream class, and will write the passed Stone to standard output in Boulder format. This means that it is currently not possible to write a Boulder::Omim object back into Omim flatfile form. OUTPUT TAGS
The tags returned by the parsing operation are taken from the names shown in the network Entrez interface to Omim. Top-Level Tags These are tags that appear at the top level of the parsed Omim entry. Identifier The Omim identifier of this entry. Identifier is a single-value tag. Example: my $identifierNo = $s->Identifier; Title The Omim title for this entry. Example: my $titledef=$s->Title; Text The Text of this Omim entry Example: my $thetext=$s->Text; Mini The text condensed version, also called "Mini" in Entrez interface Example: my $themini=$s->Mini; SeeAlso References to other relevant work. Example: my $thereviews=$s->Reviews; CreationDate This field contains the name of the person who originated the initial entry in OMIM and the date it appeared in the database. The entry may have been subsequently added to, edited, or totally rewritten by others, and their attribution is listed in the CONTRIBUTORS field. Example: my $theCreation=$s->CreationDate; Contributors This field contains a list, in chronological order, of the persons who have contributed significantly to the content of the MIM entry. The name is followed by "updated", "edited" or "re-created". Example: my @theContributors=$s->Contributors; History This field contains the edit history of this record, with an identifier and a date in which minor changes had been performed on the record. Example: my @theHistory=$s->History; References The references cited in the entry. Example: my @theReferences=$s->References; ClinicalSynopsis The content of the Clinical Synopsis data field. Example: my @theClinicalSynopsis=$s->ClinicalSynopsis; AllelicVariants The Allelic Variants Example: my @theAllelicVariants=$s->AllelicVariants; SEE ALSO
Boulder, Boulder::Blast, Boulder::Genbank AUTHOR
Lincoln Stein <lstein@cshl.org>. Luca I.G. Toldo <luca.toldo@merck.de> Copyright (c) 1997 Lincoln D. Stein Copyright (c) 1999 Luca I.G. Toldo This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See DISCLAIMER.txt for disclaimers of warranty. perl v5.10.1 2000-01-18 Boulder::Omim(3pm)
All times are GMT -4. The time now is 11:11 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy