Shell Programming and Scripting

I am new to perl. I have adopted some perl scripts that seem not to work 100%. I 've corrected such things as file sharing contention etc.

What these perl scripts basically do is extract specific records for statistical reporting of authentication and authorization from syslog daemon local logs.

now they create their own working files to perfom reporing from.

Now, these extraction scripts when they create these working files do not insert the complete line in the file. it appears to be cut off at around 180 characters.

Is this an inherent feature of perl ?

What can I do to correct this ?

thanks.

Definitely not. Perl itself poses no restriction to length of a line, I'm not sure if OS poses a restriction though. With a simple test I did on my Linux system I have been able to extract lines of 2000 characters without problems.

It is difficult to suggest a fix without seeing at least part of the actual code ....

Thanks!

Again - it could be the parsing logic. I still am trying to figure out /understand Perl itself. Here is some code. Maybe I will break it up and give you a little at a time.

my $date = $ARGV[0];
shift @ARGV;
my $errfile = "/radius/stats/errors.$date";
my $logfile = "/radius/log/local4.$date";
my $msgfile = "/radius/stats/log_msg.$date";
my $statsfile = "/radius/stats/stats.$date";
my $regfile = "/radius/stats/regid.$date";
my $ldapfile = "/radius/stats/ldap.$date";
my ( $host, $ip, $key, $nas, $reject, $server, $service, $user );
my ( %err, %nas, %regid, %stats );

$regid{'legacy'}{'accepts'}{'west'} = 0;
$regid{'legacy'}{'accepts'}{'central'} = 0;
$regid{'legacy'}{'accepts'}{'north'} = 0;
$regid{'legacy'}{'accepts'}{'northeast'} = 0;
$regid{'legacy'}{'rejects'}{'west'} = 0;
$regid{'legacy'}{'rejects'}{'central'} = 0;
$regid{'legacy'}{'rejects'}{'north'} = 0;
$regid{'legacy'}{'rejects'}{'northeast'} = 0;

#open LOG, "<$logfile" or die "$! : Unable to open $logfile for reading...\n";
open MSG, ">$msgfile" or die "Unable to open $msgfile for writing...\n";
while (defined($_ = <>)) {
if (/^\S+\s+\S+\s+\S+ (\S+?)\.\S+ .+ Protocol: Request from (\S+?) .+$/) {

$host = $1;
$nas = $2;
if ($nas =~ /^bras|^rback|^sms/) {
$service = 'adsl';
}
else {
$service = 'dial';
}

$nas{$service}{$nas}++;

if (/User (\S+) accepted $/) {
$stats{$host}{$service}{'attempts'}++;
$stats{$host}{$service}{'accepts'}++;

$user = $1;

if ($user =~ /^sbcyahooreg\@sbcglobal\.net$/i) {
if ($host =~ /anhmca|irvnca|lsanca|pltnca|snfcca|sntcca/) {
$regid{'sbcy'}{'accepts'}{'west'}++;
}
elsif ($host =~ /hstntx|kscymo|rcsntx/) {
$regid{'sbcy'}{'accepts'}{'central'}++;
}
elsif ($host =~ /chcgil|ipltin|klmzmi|milwwi|sfldmi|wotnoh/) {
$regid{'sbcy'}{'accepts'}{'north'}++;
}
elsif ($host =~ /mrdnct/) {
$regid{'sbcy'}{'accepts'}{'northeast'}++;
}
else {
$regid{'sbcy'}{'accepts'}{'unknown'}++;
}
} elsif ($user =~ /^dslreguser\@\S+$|^dslreguser$/i) {
if ($host =~ /anhmca|irvnca|lsanca|pltnca|snfcca|sntcca/) {
$regid{'legacy'}{'accepts'}{'west'}++;
}
elsif ($host =~ /hstntx|kscymo|rcsntx/) {
$regid{'legacy'}{'accepts'}{'central'}++;
}
elsif ($host =~ /chcgil|ipltin|klmzmi|milwwi|sfldmi|wotnoh/) {
$regid{'legacy'}{'accepts'}{'north'}++;
}
elsif ($host =~ /mrdnct/) {
$regid{'legacy'}{'accepts'}{'northeast'}++;
}
else {
$regid{'legacy'}{'accepts'}{'unknown'}++;
}
}
}


Do you see anything that may cause my problem ?

Where are your print statements that are writing to the files? That would help.

elsif (/User (.*) rejected \((\S+)\)/) {

if (!$user) {
$user = 'NULL';
}
$user = $1;
$reject = $2;
$stats{$host}{$service}{'attempts'}++;
$stats{$host}{$service}{$reject}++;

if ($reject eq 'ReasonNotGiven') {
$stats{$host}{$service}{'ReasonNotGiven'}++;
}

if ($user =~ /^hooreg\@\.net$/i) {
if ($host =~ /anhmca|irvnca|lsanca|pltnca|snfcca|sntcca/) {
$regid{'sbcy'}{'rejects'}{'west'}++;
}
elsif ($host =~ /hstntx|kscymo|rcsntx/) {
$regid{'sbcy'}{'rejects'}{'central'}++;
}
elsif ($host =~ /chcgil|ipltin|klmzmi|milwwi|sfldmi|wotnoh/) {
$regid{'sbcy'}{'rejects'}{'north'}++;
}
elsif ($host =~ /mrdnct/) {
$regid{'sbcy'}{'rejects'}{'northeast'}++;
}
else {
$regid{'sbcy'}{'rejects'}{'unknown'}++;
}
}
elsif ($user =~ /^dslreguser\@\S+$|^dslreguser$/i) {
if ($host =~ /anhmca|irvnca|lsanca|pltnca|snfcca|sntcca/) {
$regid{'legacy'}{'rejects'}{'west'}++;
}
elsif ($host =~ /hstntx|kscymo|rcsntx/) {
$regid{'legacy'}{'rejects'}{'central'}++;
}
elsif ($host =~ /chcgil|ipltin|klmzmi|milwwi|sfldmi|wotnoh/) {
$regid{'legacy'}{'rejects'}{'north'}++;
}
elsif ($host =~ /mrdnct/) {
$regid{'legacy'}{'rejects'}{'northeast'}++;
}
else {
$regid{'legacy'}{'rejects'}{'unknown'}++;
}
}
}
elsif (/Named Authentication Service|Could not identify Authorization Service/) {
next;
}
else {
print MSG "Protocol -> $_\n";
}
}

Just from what you posted, is the final

print MSG "Protocol -> ..."

the only place which gives the output in your script, as I don't know if this is the whole script?

It is not obvious what the problem is. I guess it has more to do with incomplete extraction with regular expressions (which is also related to your data set). Also, your program has uses of $_ which I usually discourage because it makes the string you are operating on difficult to identify. The not explicit use of a variable makes searching, replacement and so on fallback to $_ so after a few such operations it becomes difficult to deduce what is inside. This does not seem to be related to your problem though, but without indentation, your code posted is really tiresome to read.

Maybe if you revise the code, consider breaking it up into a number of smaller functions each of which doing a small number of things to avoid lengthy deeply nested blocks that are difficult to trace.