03-15-2013
Certainly putting the tr to delete white space first may speed things up. Doing it all in C/c++/java/perl/python/awk might help. Move dos2unix after the various reducers. What is a 'foot'? tail?
6 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I need help with a problem that I have not been able to figure out.
I have a file that is about 650K lines. Records are seperated by
blank lines, fields seperated by new lines. I was trying to make
a report that would add up 2 fields and associate them with a CP.
example output would be... (11 Replies)
Discussion started by: timj123
11 Replies
2. Shell Programming and Scripting
I have a log file that has many SQL statements/queries/blocks and their resultant output (success or failure) added to each of them. I need to pick up all the statements which caused errors and write them to a separate file.
On most cases, the SQL statement is a single line, like DROP . And if... (1 Reply)
Discussion started by: exchequer598
1 Replies
3. Shell Programming and Scripting
I don't know if this is a big issue or not, but I'm having difficulties. I apoligize for the upcoming essay :o.
I'm writing a script, similar to a paint program that edits images, but in the form of ANSI block characters. The program so far is working. I managed to save the image into a file,... (14 Replies)
Discussion started by: tinman47
14 Replies
4. Shell Programming and Scripting
hello,
i have a complex awk problem...
i have two tables, one with a value (0 to 1) and it's corresponding p-value, like this:
1. table:
______________________________
value p-value
... ...
0.254 0.003
0.245 0.005
0.233 0.006
... ...
______________________________
and a... (6 Replies)
Discussion started by: dietmar13
6 Replies
5. Shell Programming and Scripting
I have a awk script that parses many millions of lines so performance is critical. At one point I am extracting some variables from a space delimited string.
alarm = $11; len = split(alarm,a," "); ent = a; chem = a; for (i = 5; i<= len; i++) {chem = chem " " a}It works but is slow. Adding the... (7 Replies)
Discussion started by: Michael Stora
7 Replies
6. Shell Programming and Scripting
Hello fellow unix geeks,
I am having a small dilemna trying to parse a log file I have. Below is a sample of what it will look like:
MY_TOKEN1(group) TOKEN(other)|SSID1
MY_TOKEN2(group, group2)|SSID2
What I need to do is only keep the MY_TOKEN pieces and where there are multiple... (7 Replies)
Discussion started by: dagamier
7 Replies
LEARN ABOUT MOJAVE
english
English(3pm) Perl Programmers Reference Guide English(3pm)
NAME
English - use nice English (or awk) names for ugly punctuation variables
SYNOPSIS
use English;
use English qw( -no_match_vars ) ; # Avoids regex performance penalty
# in perl 5.16 and earlier
...
if ($ERRNO =~ /denied/) { ... }
DESCRIPTION
This module provides aliases for the built-in variables whose names no one seems to like to read. Variables with side-effects which get
triggered just by accessing them (like $0) will still be affected.
For those variables that have an awk version, both long and short English alternatives are provided. For example, the $/ variable can be
referred to either $RS or $INPUT_RECORD_SEPARATOR if you are using the English module.
See perlvar for a complete list of these.
PERFORMANCE
NOTE: This was fixed in perl 5.20. Mentioning these three variables no longer makes a speed difference. This section still applies if
your code is to run on perl 5.18 or earlier.
This module can provoke sizeable inefficiencies for regular expressions, due to unfortunate implementation details. If performance matters
in your application and you don't need $PREMATCH, $MATCH, or $POSTMATCH, try doing
use English qw( -no_match_vars ) ;
. It is especially important to do this in modules to avoid penalizing all applications which use them.
perl v5.18.2 2014-01-06 English(3pm)