Sponsored Content
Top Forums Shell Programming and Scripting Perl Array / pattern match large CPU usage Post 302358974 by Donkey25 on Monday 5th of October 2009 11:01:36 AM
Old 10-05-2009
Perl Array / pattern match large CPU usage

Hi,

I have one file in this format

Code:
20  value1
33   value2
56   value3

I have another file in this format:

Code:
34,30-SEP-09,57,100237775,33614510126,2,34
34,30-SEP-09,57,100237775,33620766654,2,34
34,30-SEP-09,108,100237775,33628458122,2,34
34,30-SEP-09,130,100237775,33635266741,2,254
34,30-SEP-09,135,100237775,33634650517,2,254
34,30-SEP-09,149,100237775,33660985888,2,34

What I want to do is for each value in the first column of the first file (lets call it x) I want to print the last column in the 2nd file if the 5th column begins with x.

So for the above, the first value in the first column of the first file is 20. None of the column4 values in the 2nd file begin with 20 so I don't want to print anything. For 33 from the first file I would want to print the value in the last column of the 2nd file for each line e.g 34,34,34,254,254,34.

I have this, it works but my CPU usage for this process is about 30%! Is there a way to make this more efficient?

Code:
#!/usr/local/bin/perl

use lib "/usr/local/include/modules";
use Getopt::Long;
use File::Copy;
use JDFunction;
use JDProcess;
use JDInput;
#use strict;
use FileHandle;
use IO;
use POSIX qw(strftime);
use Time::Local;

$file_name="/export/home/file2";
open(DATA, $file_name) || die ("Could not open file2!");
@my_data=<DATA>;
close(DATA);

$first_file="/export/home/file1";
open $FIRST_DATA, "<", $first_file or die "Could not open first data file";
while ( $line = <$FIRST_DATA> )
{
chomp ($line);
($code, $value) = split(/\t/, $line);

                foreach $line (@my_data)
                {
                chomp($line);
                ($id,$day,$ref,$valuey,$valuez,$evt_type,$subcode)=split(/,/,$line);

                        if ( $valuez =~ /^$code/ )
                        {
                        print "$code, $subcode\n";
                        }
                }

}
close $FIRST_DATA;

If you know a way to make this more efficient I'd be grateful!

Thanks
 

10 More Discussions You Might Find Interesting

1. HP-UX

how can I find cpu usage memory usage swap usage and logical volume usage

how can I find cpu usage memory usage swap usage and I want to know CPU usage above X% and contiue Y times and memory usage above X % and contiue Y times my final destination is monitor process logical volume usage above X % and number of Logical voluage above can I not to... (3 Replies)
Discussion started by: alert0919
3 Replies

2. AIX

How to monitor the IBM AIX server for I/O usage,memory usage,CPU usage,network..?

How to monitor the IBM AIX server for I/O usage, memory usage, CPU usage, network usage, storage usage? (3 Replies)
Discussion started by: laknar
3 Replies

3. Shell Programming and Scripting

Perl Pattern Match

Hi Friends, I have a tuff time with regular expressionss. Please let me know how to make this happen as it consumed lots of my time but in vain. Here is the sample text file i need to match for. I need to search for pattern1 removed, if it matches then search for pattern types either SE\ or... (2 Replies)
Discussion started by: nmattam
2 Replies

4. Shell Programming and Scripting

perl pattern match on xml

using perl Hi All, i was wondering if anyone can solve how to extract the full tag from the xml line ie not sure what to put in the m// to get the string "/data/TOP471//context_data/instruments.txt" I basically want the above filename in a variable for further processing... $_ =" ... (0 Replies)
Discussion started by: satnamx
0 Replies

5. Shell Programming and Scripting

PERL : Read an array and write to another array with intial string pattern checks

I have an array and two variables as below, I need to check if $datevar is present in $filename. If so, i need to replace $filename with the values in the array. I need the output inside an ARRAY How can this be done. Any help will be appreciated. Thanks in advance. (2 Replies)
Discussion started by: irudayaraj
2 Replies

6. Shell Programming and Scripting

Match Pattern and store next value into array

Hi, I am trying to write a script which parses a log file and will eventually put the values in an array so that I can perform some math on it. In this file I am only interested in the last 200 lines so here is the command I use to display the contents in a manageable manner. tail -200... (3 Replies)
Discussion started by: Filter500
3 Replies

7. HP-UX

Perl script limit cpu usage

Hi Experts, I am executing multiple instances(in parallel) of perl script on HP-UX box. OS is allocating substantial amount of CPU to these perl processes,resulting higher cpu utilization. Glance always shows perl processes are occupying majority of the CPU resource. It is causing slower... (2 Replies)
Discussion started by: sai_2507
2 Replies

8. Shell Programming and Scripting

Perl match pattern

Hi all, i have a peice of Perl script like this: foreach (@line) { @tmp = split /;/,$_; #print "Line is: $_\n"; switch($tmp){ case m/p60/i { push @p60, , $tmp ]; ... (7 Replies)
Discussion started by: arrals_vl
7 Replies

9. Shell Programming and Scripting

Help needed - Split large file into smaller files based on pattern match

Help needed urgently please. I have a large file - a few hundred thousand lines. Sample CP START ACCOUNT 1234556 name 1 CP END ACCOUNT CP START ACCOUNT 2224444 name 1 CP END ACCOUNT CP START ACCOUNT 333344444 name 1 CP END ACCOUNT I need to split this file each time "CP START... (7 Replies)
Discussion started by: frustrated1
7 Replies

10. Shell Programming and Scripting

Perl : Large amount of data put into an array

This basic code works. I have a very long list, almost 10000 lines that I am building into the array. Each line has either 2 or 3 fields as shown in the code snippit. The array elements are static (for a few reasons that out of scope of this question) the list has to be "built in". It... (5 Replies)
Discussion started by: sumguy
5 Replies
CTRSNA(l)								 )								 CTRSNA(l)

NAME
CTRSNA - estimate reciprocal condition numbers for specified eigenvalues and/or right eigenvectors of a complex upper triangular matrix T (or of any matrix Q*T*Q**H with Q unitary) SYNOPSIS
SUBROUTINE CTRSNA( JOB, HOWMNY, SELECT, N, T, LDT, VL, LDVL, VR, LDVR, S, SEP, MM, M, WORK, LDWORK, RWORK, INFO ) CHARACTER HOWMNY, JOB INTEGER INFO, LDT, LDVL, LDVR, LDWORK, M, MM, N LOGICAL SELECT( * ) REAL RWORK( * ), S( * ), SEP( * ) COMPLEX T( LDT, * ), VL( LDVL, * ), VR( LDVR, * ), WORK( LDWORK, * ) PURPOSE
CTRSNA estimates reciprocal condition numbers for specified eigenvalues and/or right eigenvectors of a complex upper triangular matrix T (or of any matrix Q*T*Q**H with Q unitary). ARGUMENTS
JOB (input) CHARACTER*1 Specifies whether condition numbers are required for eigenvalues (S) or eigenvectors (SEP): = 'E': for eigenvalues only (S); = 'V': for eigenvectors only (SEP); = 'B': for both eigenvalues and eigenvectors (S and SEP). HOWMNY (input) CHARACTER*1 = 'A': compute condition numbers for all eigenpairs; = 'S': compute condition numbers for selected eigenpairs specified by the array SELECT. SELECT (input) LOGICAL array, dimension (N) If HOWMNY = 'S', SELECT specifies the eigenpairs for which condition numbers are required. To select condition numbers for the j-th eigenpair, SELECT(j) must be set to .TRUE.. If HOWMNY = 'A', SELECT is not referenced. N (input) INTEGER The order of the matrix T. N >= 0. T (input) COMPLEX array, dimension (LDT,N) The upper triangular matrix T. LDT (input) INTEGER The leading dimension of the array T. LDT >= max(1,N). VL (input) COMPLEX array, dimension (LDVL,M) If JOB = 'E' or 'B', VL must contain left eigenvectors of T (or of any Q*T*Q**H with Q unitary), corresponding to the eigenpairs specified by HOWMNY and SELECT. The eigenvectors must be stored in consecutive columns of VL, as returned by CHSEIN or CTREVC. If JOB = 'V', VL is not referenced. LDVL (input) INTEGER The leading dimension of the array VL. LDVL >= 1; and if JOB = 'E' or 'B', LDVL >= N. VR (input) COMPLEX array, dimension (LDVR,M) If JOB = 'E' or 'B', VR must contain right eigenvectors of T (or of any Q*T*Q**H with Q unitary), corresponding to the eigenpairs specified by HOWMNY and SELECT. The eigenvectors must be stored in consecutive columns of VR, as returned by CHSEIN or CTREVC. If JOB = 'V', VR is not referenced. LDVR (input) INTEGER The leading dimension of the array VR. LDVR >= 1; and if JOB = 'E' or 'B', LDVR >= N. S (output) REAL array, dimension (MM) If JOB = 'E' or 'B', the reciprocal condition numbers of the selected eigenvalues, stored in consecutive elements of the array. Thus S(j), SEP(j), and the j-th columns of VL and VR all correspond to the same eigenpair (but not in general the j-th eigenpair, unless all eigenpairs are selected). If JOB = 'V', S is not referenced. SEP (output) REAL array, dimension (MM) If JOB = 'V' or 'B', the estimated reciprocal condition numbers of the selected eigenvectors, stored in consecutive elements of the array. If JOB = 'E', SEP is not referenced. MM (input) INTEGER The number of elements in the arrays S (if JOB = 'E' or 'B') and/or SEP (if JOB = 'V' or 'B'). MM >= M. M (output) INTEGER The number of elements of the arrays S and/or SEP actually used to store the estimated condition numbers. If HOWMNY = 'A', M is set to N. WORK (workspace) COMPLEX array, dimension (LDWORK,N+1) If JOB = 'E', WORK is not referenced. LDWORK (input) INTEGER The leading dimension of the array WORK. LDWORK >= 1; and if JOB = 'V' or 'B', LDWORK >= N. RWORK (workspace) REAL array, dimension (N) If JOB = 'E', RWORK is not referenced. INFO (output) INTEGER = 0: successful exit < 0: if INFO = -i, the i-th argument had an illegal value FURTHER DETAILS
The reciprocal of the condition number of an eigenvalue lambda is defined as S(lambda) = |v'*u| / (norm(u)*norm(v)) where u and v are the right and left eigenvectors of T corresponding to lambda; v' denotes the conjugate transpose of v, and norm(u) denotes the Euclidean norm. These reciprocal condition numbers always lie between zero (very badly conditioned) and one (very well condi- tioned). If n = 1, S(lambda) is defined to be 1. An approximate error bound for a computed eigenvalue W(i) is given by EPS * norm(T) / S(i) where EPS is the machine precision. The reciprocal of the condition number of the right eigenvector u corresponding to lambda is defined as follows. Suppose T = ( lambda c ) ( 0 T22 ) Then the reciprocal condition number is SEP( lambda, T22 ) = sigma-min( T22 - lambda*I ) where sigma-min denotes the smallest singular value. We approximate the smallest singular value by the reciprocal of an estimate of the one-norm of the inverse of T22 - lambda*I. If n = 1, SEP(1) is defined to be abs(T(1,1)). An approximate error bound for a computed right eigenvector VR(i) is given by EPS * norm(T) / SEP(i) LAPACK version 3.0 15 June 2000 CTRSNA(l)
All times are GMT -4. The time now is 06:15 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy