01-24-2011
@vsachan
Having re-read my post I was a bit blunt yesterday. On my computer I can't unzip a file with the wrong file extension.
I have written one-off searches of large numbers of reasonably large compressed text files in Shell and realise the practical limits. On your scale this is not a job for Shell programming.
This data came from somewhere and I would be very surprised if it only ever existed as a flat file. I suppose that you might be trying to find data rejections in context?
I am very concerned that this now appears to be a fuzzy search. The length of the search string and the position and size of the searchable data varies between your posts. This makes the software design so much more difficult and I withdraw any implication that this task is feasible.
On more reflection strongly advise that you get a Systems Analyst and a Database Designer on this job with view to possibly using a database approach. To my mind it is too early to engage a Database Programmer.
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
I have one file:
123*100*abcd*10
123*101*abcd*-29*def
123*100*abcd*-10
123*102*abcd*-105*asd
I would like to parameterize the search patterns in the following way so that the user could dynamically change the search pattern.
*100* and *- (ie *minus)
*102* and *-
The output that is... (6 Replies)
Discussion started by: augustinep
6 Replies
2. Shell Programming and Scripting
Hi
I'm not very good with the serach patterns and I'd need a sample how to find a line that has multiple patterns.
Say I want to find a line that has "abd", "123" and "QWERTY" and there can be any characters or numbers between the serach patterns, I have a file that has thousands of lines and... (10 Replies)
Discussion started by: Juha
10 Replies
3. Shell Programming and Scripting
Good day, great gurus,
I'm new to Perl, and programming in general. I'm trying to retrieve a column of data from my text file which spans a non-specific number of lines. So I did a regexp that will pick out the columns. However,my pattern would vary. I tried using a foreach loop unsuccessfully.... (2 Replies)
Discussion started by: Sp3ck
2 Replies
4. Shell Programming and Scripting
I have two lists in a file that look like
a b
b a
e f
c d
f e
d c
I would like a final list
a b
c d
e f
I've tried multiple grep and awk but can't get it to work (8 Replies)
Discussion started by: godzilla07
8 Replies
5. Shell Programming and Scripting
Hi,
I tried to search multiple pattern using awk
trans=1234
reason=LN MISMATCH
rec=`awk '/$trans/ && /'"$reason"'/' file`
whenevr i tried to run on command promt it is executing but when i tried to implment same logic in shell script,it is failing i.e $rec is empty
... (6 Replies)
Discussion started by: ns64110
6 Replies
6. Shell Programming and Scripting
I have a list of files all over a file system e.g.
/home/1/foo/bar.x
/www/sites/moose/foo.txtI'm looking for strings in these files and want to replace each occurrence with a replacement string, e.g.
if I find: '#@!^\&@ in any of the files I want to replace it with: 655#@11, etc.
There... (2 Replies)
Discussion started by: spacegoose
2 Replies
7. Shell Programming and Scripting
Hi,
I want to grep multiple patterns from multiple files and save to multiple outputs. As of now its outputting all to the same file when I use this command.
Input : 108 files to check for 390 patterns to check for. output I need to 108 files with the searched patterns.
Xargs -I {} grep... (3 Replies)
Discussion started by: Diya123
3 Replies
8. Shell Programming and Scripting
Hi All,
I am starting a service which will redirect its out put into 2 logs say A and B.
Now for succesful startup of the service i need to search pattern1 in log A and pattern2 in log B which are writen continuosly.
Now my requirement is to find the patterns in the increasing logs A and B... (19 Replies)
Discussion started by: Girish19
19 Replies
9. Shell Programming and Scripting
Hi,
I have scenario like below and need to search for multiple patterns
Eg:
Test
Time Started= secs
Time Ended = secc
Green test
Test
Time Started= secs
Time Ended = secc
Green test
Output:
I need to display the text starting with Test and starting with Time... (2 Replies)
Discussion started by: weknowd
2 Replies
10. UNIX for Beginners Questions & Answers
Im having an issue when trying to replace the first column with a new set of values in multiple files. The results from the following code only replaces the files with the last set of values in val.txt. I want to replace all the files with all the values.
for date in {1..31}
do
for val in... (1 Reply)
Discussion started by: ncwxpanther
1 Replies
LEARN ABOUT DEBIAN
bp_search2tribe
BP_SEARCH2TRIBE(1p) User Contributed Perl Documentation BP_SEARCH2TRIBE(1p)
NAME
search2tribe - Turn SearchIO parseable reports(s) into TRIBE matrix
SYNOPSIS
Usage:
search2tribe [-o outputfile] [-f reportformat] [-w/--weight] file1 file2 ..
DESCRIPTION
This script is probably too slow for most people's uses. It is better to use something like scripts/searchio/fastam9_to_table, -m 9 output
from BLAST, or the blast2table from the BLAST O'Reilly book to get a tabular output from these programs and then feed the table into MCL
with the mcxdeblast script and the --m9 option.
This script will turn a protein Search report (BLASTP, FASTP, SSEARCH) into a Markov Matrix for TribeMCL clustering.
The options are:
-o filename - the output filename [default STDOUT]
-f format - search result format (blast, fasta)
(ssearch is fasta format). default is blast.
-w or --weight VALUE - Change the default weight for E(0.0) hits
to VALUE (default=200 (i.e. 1e-200) )
-h - this help menu
Additionally specify the filenames you want to process on the command-line. If no files are specified then STDIN input is assumed. You
specify this by doing: search2tribe < file1 file2 file3
AUTHOR
Jason Stajich, jason-at-bioperl-dot-org
perl v5.14.2 2012-03-02 BP_SEARCH2TRIBE(1p)