Using perl to obtain stats instead of grep Post: 302415071

Sponsored Content

Top Forums Shell Programming and Scripting Using perl to obtain stats instead of grep Post 302415071 by jeffersno1 on Wednesday 21st of April 2010 04:35:46 PM

04-21-2010

Registered User

Using perl to obtain stats instead of grep

Evening all, i hope you guys can help !!

I'm working on a pass purchase system and would like to run a script every day from cron which looks at the previous day log files and produce some kind of stats for passes purchased / success / failures.

Issues.

I have around 25 passes and for each pass im running a grep for the SUBS_BOOK_ACC followed by an egrep for success and failure. Each one i put into a variable and print at the end. It works but i'm concerned im wasting resources on a system which is highly utilised.

Code:

grep SUBS_BOOK_ACC bkpass_log_2010042000000.audit | egrep `EPASS088|success` | wc -l

some sample data from the log file

2010-04-20_00:01:02:307,SUBS_BOOK_ACC,system@bkpass_1271_718059_59480,441234567899,EPASS088,,success,9999
2010-04-20_00:01:18:139,SUBS_BOOK_ACC,system@bkpass_1271_718076_442346,441234567899,EPASS088,,success,6585
2010-04-20_00:01:24:335,SUBS_BOOK_ACC,system@bkpass_1271_718080_786699,441234567899,EPASSV09,,PAY payment failed,211

ideal output would be something like

Pass ID Success Failures

EPASSV09 7338 21
EPASS088 3852 11
EPASS013 452 55

I understand that i can use perl to pull the data and inspect the file line by line, instead of using multiple greps.

I've started a perl script but im having issues understand the syntax. I've input the file but im struggling with the pattern matching. Can someone help me understand how i can inspect each line and search for a sting, count and print.?

Can someone please help. i understand chomp is used to split the line and that i split the line at the , mark then i try the pattern match

Code:

#!/usr/bin/perl -w
use POSIX;
 
$LOGFILE = "/export/home/user/stats.test";
open (LOGFILE) or die ("Could not open log file.");
 
foreach $line (<LOGFILE>){
 
        chomp($_);
        @line=split(",",$_);
        @time=split(",",$line[0]);
        if (( $line[0] = m/SUBS_BOOK_ACC/ )){
                $results{"$time[0]:$time[1]"}[0]++;
        }
        $results{"$time[0]:$time[1]"}[1]++;
print($results);

In the above code im trying to find out how many pas booking were attempted.

If anyone can help i'd be eternally grateful.

Many thanks

Jeffers

jeffersno1

View Public Profile for jeffersno1

Find all posts by jeffersno1

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Newbie to perl - Help with stats script

Hi, this is my first post so here goes..... I need help... I am trying to write a script to produce some stats based on a number of searches in a log file. Now i know how to do this using multiple variables which are really just greps, but I want a more efficent way of doing this as my poor...

2. Shell Programming and Scripting

grep using Perl

I'm using perl to do a grep of each line in a vendor file and find its occurrences in a specific directory. Any values found is saved in @dir. .....(file opened, etc.) .... while ($line=<FILE>){ @dir = `grep $line * `; } It's the specific usage of the system grep that I'm having...

3. Shell Programming and Scripting

Perl grep

OK here's the situation: I have got these lines which I have got to parse. If the line contains a particular string and any element from a previously defined array I need to take that particular line and do some further processing. if ((grep(/$_/,$1)) && (grep($pattern,@myarr))) { #Do...

4. Shell Programming and Scripting

How to execute Grep in Perl.

$ grep edge test_1 |sort|uniq >result.txt $more result.txt edge-a-pas01.com 10.12.10.11 edge-b-pas02.com 10.12.10.12 edge-c-pas03.com 10.12.10.50 edge-d-pas03.com 10.12.10.10 how do we execute the above grep command using perl? Thanks in advance.

5. Shell Programming and Scripting

grep in perl

Hello I want to grep a line from a file saved in some directory. Can anyone please correct the code below: #!/usr/bin/perl -w $file = "/home/output.txt" $grep_line = "closing zip for topic"; `grep $grep_line* $file`;

6. Shell Programming and Scripting

Perl + and Grep

Hi All i have this script that uses glob to look in /var/log/messages.* my @messagefiles = glob "/var/log/messages.*"; and the code that uses it is this grep { /NVRM: Xid/ } @messages) but this spits out this /var/log/messages-20111030:Oct 25 13:43:04 brent kernel: NVRM:...

7. Shell Programming and Scripting

GREP Issue in Perl

Im storing multiple functions in a varaible called $check... The variable check contains the following: a() b() c() ... ..etc now im checking individually which function is kept in which file using GREP if ( grep \$check \i, <FILE> ) The problem is im getting the output for the...

8. Shell Programming and Scripting

Grep in PERL

Hi, Can anybody let me know how this grep will work. The input and output is not known. Also can you give me the details of any link where i can find clearly about grep Thanks in advance

9. Shell Programming and Scripting

How to grep a pattern in perl?

hello Everyone i am a newbie. i have a file which contains the following E:\gtmproj\script\i486_nt\obj\check_geomtools.exe: o:\portsrc\spg\system_1\i486_nt\advapps\TK-2\objmt\winclockmtq.lib E:\gtmproj\script\i486_nt\obj\check_geomtools.exe:...

10. Shell Programming and Scripting

Need help in solving to obtain desired print output using awk or perl or any commands, Please help!!

I have an file which have data in lines as follows ad, findline=24,an=54,ab=34,av=64,ab=7989,ab65=34,aj=323,ay=34,au=545,ad=5545 ab,abc,an10=23,an2=24,an31=32,findline=00,an33=23,an32=26,an40=45,ac23=5,ac=87,al=76,ad=26...

LEARN ABOUT OPENDARWIN

split

SPLIT(1)						    BSD General Commands Manual 						  SPLIT(1)

NAME

     split -- split a file into pieces

SYNOPSIS

     split [-a suffix_length] [-b byte_count[k|m]] [-l line_count] [-p pattern] [file [name]]

DESCRIPTION

     The split utility reads the given file and breaks it up into files of 1000 lines each.  If file is a single dash ('-') or absent, split reads
     from the standard input.

     The options are as follows:

     -a      Use suffix_length letters to form the suffix of the file name.

     -b      Create smaller files byte_count bytes in length.  If ``k'' is appended to the number, the file is split into byte_count kilobyte
	     pieces.  If ``m'' is appended to the number, the file is split into byte_count megabyte pieces.

     -l      Create smaller files n lines in length.

     -p pattern
	     The file is split whenever an input line matches pattern, which is interpreted as an extended regular expression.	The matching line
	     will be the first line of the next output file.  This option is incompatible with the -b and -l options.

     If additional arguments are specified, the first is used as the name of the input file which is to be split.  If a second additional argument
     is specified, it is used as a prefix for the names of the files into which the file is split.  In this case, each file into which the file is
     split is named by the prefix followed by a lexically ordered suffix using suffix_length characters in the range ``a-z''.  If -a is not speci-
     fied, two letters are used as the suffix.

     If the name argument is not specified, the file is split into lexically ordered files named with prefixes in the range of ``x-z'' and with
     suffixes as above.

SEE ALSO

     csplit(1), re_format(7)

STANDARDS

     The split utility conforms to IEEE Std 1003.1-2001 (``POSIX.1'').

HISTORY

     A split command appeared in Version 3 AT&T UNIX.

BUGS

     For historical reasons, if you specify name, split can only create 676 separate files.  The default naming convention allows 2028 separate
     files.  The -a option can be used to work around this limitation.

     The maximum line length for matching patterns is 65536.

BSD
								  April 16, 1994							       BSD