01-16-2015
Quote:
Originally Posted by
yifangt
My problem is the implementation. I want to step to "my second stage" of programming by using those available libraries.
"hash table" isn't exactly a library, it's different enough from other data structures it's often hand-rolled. Generalizing it too much would run the risk of poor performance, you need to pick the right algorithms for your application. It has a lot of restrictions as well (hard to iterate, deletion can cause something like fragmentation, and it can't be sorted). I've seen a few attempts at building a library for it, but nothing I ever liked very much.
In the end it's not that complicated. It's a big array with strict rules about what data gets put in what element. I'd suggest "open chaining" for your table -- basically an array full of lists -- with an index that's not really hashed at all, just converted from ACGT into boolean. Four letters would be 8 bits, for an array 256 long for example. Then you could just look up the first four letters of your sequence, find that list, and speedily check every possible thing which might contain your sequence without having to brute-force it.
Last edited by Corona688; 01-16-2015 at 11:24 AM..
This User Gave Thanks to Corona688 For This Post:
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
Hi ,
i'm searching for files over many Aix servers with rsh command using this request :
find /dir1 -name '*.' -exec ls {} \;
and then count them with "wc"
but i would improve this search because it's too long and replace directly find with ls command but "ls *. " doesn't work.
and... (3 Replies)
Discussion started by: Nicol
3 Replies
2. Shell Programming and Scripting
Hi All,
I am using grep command to find string "abc" in one file .
content of file is
***********
abc = xyz
def= lmn
************
i have given the below mentioned command to redirect the output to tmp file
grep abc file | sort -u | awk '{print #3}' > out_file
Then i am searching... (2 Replies)
Discussion started by: pooga17
2 Replies
3. UNIX for Dummies Questions & Answers
hi someone tell me which ways i can improve disk I/O and system process performance.kindly refer some commands so i can do it on my test machine.thanks, Mazhar (2 Replies)
Discussion started by: mazhar99
2 Replies
4. Shell Programming and Scripting
I have a data file of 2 gig
I need to do all these, but its taking hours, any where i can improve performance, thanks a lot
#!/usr/bin/ksh
echo TIMESTAMP="$(date +'_%y-%m-%d.%H-%M-%S')"
function showHelp {
cat << EOF >&2
syntax extreme.sh FILENAME
Specify filename to parse
EOF... (3 Replies)
Discussion started by: sirababu
3 Replies
5. Shell Programming and Scripting
Hi Friends,
I wrote the below shell script to generate a report on alert messages recieved on a day. But i for processing around 4500 lines (alerts) the script is taking aorund 30 minutes to process.
Please help me to make it faster and improve the performace of the script. i would be very... (10 Replies)
Discussion started by: apsprabhu
10 Replies
6. Shell Programming and Scripting
Hi All,
I have written a script as follows which is taking lot of time in executing/searching only 3500 records taken as input from one file in log file of 12 GB Approximately.
Working of script is read the csv file as an input having 2 arguments which are transaction_id,mobile_number and search... (6 Replies)
Discussion started by: poweroflinux
6 Replies
7. Shell Programming and Scripting
Hi,
I have around one lakh records. I have used XML for the creation of the data.
I have used these 2 Perl modules.
use XML::DOM;
use XML::LibXML;
The data will loo like this and most it is textual entries.
<eid>19000</eid>
<einfo>This is the ..........</einfo>
......... (3 Replies)
Discussion started by: vanitham
3 Replies
8. Programming
Input file:
#content_1
12314345345
242467
#content_14
436677645
576577657
#content_100
3425546
56
#content_12
243254546
1232454
.
.
Reference file:
content_100 (1 Reply)
Discussion started by: cpp_beginner
1 Replies
9. Shell Programming and Scripting
Hi,
I have a script which looks like this.
Input file
data1^20
data2^30
#!/bin/sh
file"/home/Test.txt"
while read line
do
echo $line |awk 'BEGIN { FS = "^" } ; { print $2 }'
echo $line |awk 'BEGIN { FS = "^" } ; { print $1 }' | gzip | wc -c
done <"$file"
How can i... (4 Replies)
Discussion started by: chetan.c
4 Replies
10. UNIX for Dummies Questions & Answers
Hi ,
i wrote a script to convert dates to the formate i want .it works fine but the conversion is tkaing lot of time . Can some one help me tweek this script
#!/bin/bash
file=$1
ofile=$2
cp $file $ofile
mydates=$(grep -Po '+/+/+' $ofile) # gets 8/1/13
mydates=$(echo "$mydates" | sort |... (5 Replies)
Discussion started by: vikatakavi
5 Replies
LEARN ABOUT DEBIAN
bio::align::utilities
Bio::Align::Utilities(3pm) User Contributed Perl Documentation Bio::Align::Utilities(3pm)
NAME
Bio::Align::Utilities - A collection of utilities regarding converting and manipulating alignment objects
SYNOPSIS
use Bio::Align::Utilities qw(:all);
# %dnaseqs is a hash of CDS sequences (spliced)
# Even if the protein alignments are local make sure the start/end
# stored in the LocatableSeq objects are to the full length protein.
# The CoDing Sequence that is passed in should still be the full
# length CDS as the nt alignment will be generated.
#
my $dna_aln = &aa_to_dna_aln($aa_aln,\%dnaseqs);
# generate bootstraps
my $replicates = &bootstrap_replicates($aln,$count);
DESCRIPTION
This module contains utility methods for manipulating sequence alignments ( Bio::Align::AlignI) objects.
The aa_to_dna_aln utility is essentially the same as the mrtrans program by Bill Pearson available at
ftp://ftp.virginia.edu/pub/fasta/other/mrtrans.shar. Of course this is a pure-perl implementation, but just to mention that if anything
seems odd you can check the alignments generated against Bill's program.
FEEDBACK
Mailing Lists
User feedback is an integral part of the evolution of this and other Bioperl modules. Send your comments and suggestions preferably to the
Bioperl mailing list. Your participation is much appreciated.
bioperl-l@bioperl.org - General discussion
http://bioperl.org/wiki/Mailing_lists - About the mailing lists
Support
Please direct usage questions or support issues to the mailing list:
bioperl-l@bioperl.org
rather than to the module maintainer directly. Many experienced and reponsive experts will be able look at the problem and quickly address
it. Please include a thorough description of the problem with code and data examples if at all possible.
Reporting Bugs
Report bugs to the Bioperl bug tracking system to help us keep track of the bugs and their resolution. Bug reports can be submitted via the
web:
https://redmine.open-bio.org/projects/bioperl/
AUTHOR - Jason Stajich
Email jason-at-bioperl-dot-org
APPENDIX
The rest of the documentation details each of the object methods. Internal methods are usually preceded with a _
aa_to_dna_aln
Title : aa_to_dna_aln
Usage : my $dnaaln = aa_to_dna_aln($aa_aln, \%seqs);
Function: Will convert an AA alignment to DNA space given the
corresponding DNA sequences. Note that this method expects
the DNA sequences to be in frame +1 (GFF frame 0) as it will
start to project into coordinates starting at the first base of
the DNA sequence, if this alignment represents a different
frame for the cDNA you will need to edit the DNA sequences
to remove the 1st or 2nd bases (and revcom if things should be).
Returns : Bio::Align::AlignI object
Args : 2 arguments, the alignment and a hashref.
Alignment is a Bio::Align::AlignI of amino acid sequences.
The hash reference should have keys which are
the display_ids for the aa
sequences in the alignment and the values are a
Bio::PrimarySeqI object for the corresponding
spliced cDNA sequence.
See also: Bio::Align::AlignI, Bio::SimpleAlign, Bio::PrimarySeq
bootstrap_replicates
Title : bootstrap_replicates
Usage : my $alns = &bootstrap_replicates($aln,100);
Function: Generate a pseudo-replicate of the data by randomly
sampling, with replacement, the columns from an alignment for
the non-parametric bootstrap.
Returns : Arrayref of L<Bio::SimpleAlign> objects
Args : L<Bio::SimpleAlign> object
Number of replicates to generate
cat
Title : cat
Usage : $aln123 = cat($aln1, $aln2, $aln3)
Function : Concatenates alignment objects. Sequences are identified by id.
An error will be thrown if the sequence ids are not unique in the
first alignment. If any ids are not present or not unique in any
of the additional alignments then those sequences are omitted from
the concatenated alignment, and a warning is issued. An error will
be thrown if any of the alignments are not flush, since
concatenating such alignments is unlikely to make biological
sense.
Returns : A new Bio::SimpleAlign object
Args : A list of Bio::SimpleAlign objects
perl v5.14.2 2012-03-02 Bio::Align::Utilities(3pm)