Sponsored Content
Top Forums Shell Programming and Scripting mail log parsing script in need of makeover Post 302194907 by era on Wednesday 14th of May 2008 01:55:56 AM
Old 05-14-2008
Quote:
Originally Posted by jjamd64
As always any comments, criticisms, and questions are welcome and appreciated.
Please use code tags for legibility. Fortunately I get proper indentation when I quote your message so it's not completely unreadable.

Code:
use locale;
use DBI;
use Cwd ;

You don't seem to be using the DBI stuff or Cwd; also doubtful what the locale is for. The logs don't use locale-dependent formatting, do they?

Code:
foreach $myQID (keys %sender_emails)
{
        $myto = $recipient_emails{$myQID} ;
        $myfrom = $sender_emails{$myQID} ;
        $tocount = $recipient_count{$myQID} ;
        next if $tocount >= 6;
        foreach $rcpt_group (values %sender_emails)
        {
                ($1, $2, $3, $4, $5) = split(/\s+/, $_);
                @rcpt = ("$1", "$2", "$3", "$4", "$5");
                {
                        foreach $rcpt (@rcpt)
                        {
                                print $myfrom . "," . $rcpt . \n;
                        }
                }
        }
}

You can certainly defer all processing until you have read the whole log file, but like I suggested earlier, it might be more memory-efficient to process and forget queue entries as you see them, assuming that "removed" is a good pattern for seeing when you have a full entry.

Assigning to $1 $2 etc seems wrong, I guess you might be able to do it but that's definitely not recommended. Why don't you assign the result of the split directly to the list @rcpt anyway? If you want to restrict it to just five fields and throw away the rest, you can restrict the split, or splice away the remainder after splitting. And I guess you want to be splitting $rcpt_group, not $_.

Better yet, collect the formatted output already in the initial loop, and just don't print it if the count is too large. I wanted to avoid having two variables and needlessly collect information which was not going to be printed, so I used references to lists instead, but this is certainly a workable solution as well.

Using the string join (dot) operator on a string just for printing it is mildly inefficient, you can just print a list without first gluing together its elements into a single string.

You need to double-quote the \n to make it into a newline; as it is, it's a reference to an undefined symbol (you should have used strict after all!) with a blank print value; that's why you aren't getting any newlines in your output.

To reiterate, that's basically as if you had has an identifier UNDEFINED_SYMBOL and used a backslash to produce a reference to it: \UNDEFINED_SYMBOL. Perl is unacceptable forgiving about these things when you don't use strict and use warnings, for legacy reasons -- you should always use strict and use warnings for new scripts (I wrote "nontrivial" scripts but all scripts appear trivial when you start optimistically working on them). I confess that I too sinned against this -- much against my better conscience.

Last edited by era; 05-14-2008 at 03:08 AM.. Reason: Elaborate on what unquoted \n means and why to use strict
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell script for parsing 300mb log file..

am relatively new to Shell scripting. I have written a script for parsing a big file. The logic is: Apart from lot of other useless stuffs, there are many occurances of <abc> and corresponding </abc> tags. (All of them are properly closed) My requirement is to find a particular tag (say... (3 Replies)
Discussion started by: gurpreet470
3 Replies

2. Shell Programming and Scripting

Help with script parsing a log file

I have a large log file, which I want to first use grep to get the specific lines then send it to awk to print out the specific column and if the result is zero, don't do anything. What I have so far is: LOGDIR=/usr/local/oracle/Transcription/log ERRDIR=/home/edixftp/errors #I want to be... (3 Replies)
Discussion started by: mevasquez
3 Replies

3. Shell Programming and Scripting

Shell script to parsing log

Hi I Have log like this : 0 234: { 3 2: 04 EE 7 14: '20081114081' 23 1: 00 79 10: '38809' 91 15: '528111510010159' 143 29: 'Streaming/downloading service' 174 3: 'MTV' 179 43: 'rtsp://172.28/MTV2GO-Loop.sdp' 224 1: 05 ... (10 Replies)
Discussion started by: justbow
10 Replies

4. Shell Programming and Scripting

Performance of log parsing shell script very slow

Hello, I am an absolute newbie and whatever I've written in the shell script (below) has all been built with generous help from googling the net and this forum. Please forgive any schoolboy mistakes. Now to the qn, my input file looks like this - 2009:04:03 08:21:41:513,INFO... (7 Replies)
Discussion started by: sowmitr
7 Replies

5. Shell Programming and Scripting

Script for Parsing Log File

Working on a script that inputs an IP, parses and outputs to another file. A Sample of the log is as follows: I need the script to be able to input IP and print the data in an output file in the following format or something similar: Thanks for any help you can give me! (8 Replies)
Discussion started by: Winsarc
8 Replies

6. Shell Programming and Scripting

Log parsing script

Hello, I have a script that parses logs and sends the output via digitally signed and encrypted email. This script uses grep -v to exclude patterns in a file. The problem I have is if this is run via cron none of the pattern matching seems to occur. If I run it by hand it runs exactly as it is... (2 Replies)
Discussion started by: wpfontenot
2 Replies

7. Shell Programming and Scripting

Script for parsing vertical log into horizontal

Hi, I have log like this : And i want the output like below : I have try using awk but doesn't work awk ' /ffff /{ts=$1} f && /SectorAntenna\=1/{sa1=$3} f && /SectorAntenna\=2/{sa2=$3} f && /SectorAntenna\=3/{sa3=$3} { s= ts "|" sa1 "|" sa2 "|" sa3 print s f=0 }' (7 Replies)
Discussion started by: justbow
7 Replies

8. Shell Programming and Scripting

Issue with awk script parsing log file

Hello All, I am trying to parse a log file and i got this code from one of the good forum colleagues, However i realised later there is a problem with this awk script, being naive to awk world wanted to see if you guys can help me out. AWK script: awk '$1 ~ "^WRITER_" {p=1;next}... (18 Replies)
Discussion started by: Ariean
18 Replies

9. Shell Programming and Scripting

Bash Script - Mail Secure.log

I'm putting together a fairly simple script, to check "secure.log" for content and email the results in a cron, nightly. The script runs fine upon manual execution, it's a problem when ran in cron. This is on a Mac server. Any thoughts? #!bin/bash #Email secure.log, nightly. Subject="Secure... (6 Replies)
Discussion started by: Nvizn
6 Replies

10. Shell Programming and Scripting

Parsing a log file and creating a report script

The log file is huge and lot of information, i would like to parse and make a report . below is the log file looks like: REPORT DATE: Mon Aug 10 04:16:17 CDT 2017 SYSTEN VER: v1.3.0.9 TERMINAL TYPE: prod SYSTEM: nb11cu51 UPTIME: 04:16AM up 182 days 57 mins min MODEL, TYPE, and SN:... (8 Replies)
Discussion started by: amir07
8 Replies
All times are GMT -4. The time now is 09:06 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy