|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
save every line in log file with matched string
Post 302304611 by meggae on Monday 6th of April 2009 11:38:16 PM
04-07-2009
rikxik..
thanks for the brilliant suggestion given..i also thought the problem might be with the programming (foreach statement) as i'm not into much of the programming..
i've review the code and cut out the unnecessary loop..it works and it works great if i execute it from the terminal instead of browser..
thanks insamniac for ur ksh opinion..i would really like to consider it but i don't really have much time now to learn new things..but i currently trying the code already..it surely gives me another sight way of solving coming problems..
thanks for the brilliant suggestion given..i also thought the problem might be with the programming (foreach statement) as i'm not into much of the programming..
i've review the code and cut out the unnecessary loop..it works and it works great if i execute it from the terminal instead of browser..
thanks insamniac for ur ksh opinion..i would really like to consider it but i don't really have much time now to learn new things..but i currently trying the code already..it surely gives me another sight way of solving coming problems..
Quote:
#!/usr/bin/perl -w
use File::Basename;
use CGI;
use CGI::Carp qw(fatalsToBrowser);
use File::Path;
use File::Copy;
print "Content-type: text/html\r\n";
print "\r\n";
$accessfile = "/var/log/foo/access_log";
open (example7, "/var/log/foo/access_log") || die ("Could not open file. <br> $!");
@cisco = <example7>;
@passwd = split(' ',$new);
open(READER, $accessfile) or die "canot read log:$accessfile:$!";
@LOGFILE = <READER>;
while(<READER>) {
($machine, $date, $time, $tzone, $method, $url, $httpvers, $response, $size) = /([^\s]*) [^\s]* [^\s]* \[([^\:]*)
[^\s]*) ([^\]]*)\] \"([^\s]*) ([^\s]*) HTTP\/([^\"]*)\" (\d*) (\d*)/;
($day, $month, $year) = split('/',$date);
}
foreach $string (sort(@passwd)) {
my $path = "/path/foo/foo/$string/log";
if (! -d $path)
{
mkpath($path,755) or die "Failed to create $path: $!\n";
}
$file = "$string.txt";
if ($path =~ $string) {
foreach $line (sort(@cisco)) {
$append = 0;
if ($append){
open (OUTFILE,">$file"); if ($line =~ $string) {print OUTFILE $line;}
}else{
open (OUTFILE,">>$file"); if ($line =~ $string) {print OUTFILE $line;}
} }
close (OUTFILE);
move($file,$path) or die "Failed to copy $file: $!\n";
}
}
close (example7);
use File::Basename;
use CGI;
use CGI::Carp qw(fatalsToBrowser);
use File::Path;
use File::Copy;
print "Content-type: text/html\r\n";
print "\r\n";
$accessfile = "/var/log/foo/access_log";
open (example7, "/var/log/foo/access_log") || die ("Could not open file. <br> $!");
@cisco = <example7>;
@passwd = split(' ',$new);
open(READER, $accessfile) or die "canot read log:$accessfile:$!";
@LOGFILE = <READER>;
while(<READER>) {
($machine, $date, $time, $tzone, $method, $url, $httpvers, $response, $size) = /([^\s]*) [^\s]* [^\s]* \[([^\:]*)
[^\s]*) ([^\]]*)\] \"([^\s]*) ([^\s]*) HTTP\/([^\"]*)\" (\d*) (\d*)/;($day, $month, $year) = split('/',$date);
}
foreach $string (sort(@passwd)) {
my $path = "/path/foo/foo/$string/log";
if (! -d $path)
{
mkpath($path,755) or die "Failed to create $path: $!\n";
}
$file = "$string.txt";
if ($path =~ $string) {
foreach $line (sort(@cisco)) {
$append = 0;
if ($append){
open (OUTFILE,">$file"); if ($line =~ $string) {print OUTFILE $line;}
}else{
open (OUTFILE,">>$file"); if ($line =~ $string) {print OUTFILE $line;}
} }
close (OUTFILE);
move($file,$path) or die "Failed to copy $file: $!\n";
}
}
close (example7);
Last edited by meggae; 04-07-2009 at 12:44 AM..
|
|
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
All,
Is there any way out to display the nth line before the string is matched ???
Eg : If i have a file which has the following contents and if i want to get the
3rd line before the string is matched
a
b
c
d
e
f
if i give the input as f and lines before the match as 3 then it should... (5 Replies)
Discussion started by: helper
5 Replies
2. Shell Programming and Scripting
Hi
I want to read a file line by line and search for a particular string in each line(say for example string containing @ )and save that string into a variable.
Can someone suggest me the way to implement it.I am using K- shell
Thanks
Ishita (5 Replies)
Discussion started by: Ishita
5 Replies
3. Shell Programming and Scripting
Hi ,
I have input file and i want to extract below strings
<msisdn xmlns="">0492001956</ msisdn> => numaber inside brackets
<resCode>3000</resCode> => 3000 needs to be extracted
<resMessage>Request time
getBalances_PSM.c(37): d out</resMessage></ns2:getBalancesResponse> => the word... (14 Replies)
Discussion started by: sushmab82
14 Replies
4. UNIX for Advanced & Expert Users
Hi,
I have a query as follows :
suppose I am matching a string in a file say "start from here"
and I want to pick up 'n' number of lines () from the matched string. Is there any way to do that ?
1) going forward I want to do this for every match for the above string
2) or limit this to... (2 Replies)
Discussion started by: sumoka
2 Replies
5. Shell Programming and Scripting
Hi Experts,
I am very new to scripting and have a prb since few days and it is urgent to solve so much appreciated if u help me.
i have 2 files
file1.txt
9647810043118
9647810043126
9647810043155
9647810043161
9647810043166
9647810043185
9647810043200
9647810043203
9647810043250... (22 Replies)
Discussion started by: mustafa.abdulsa
22 Replies
6. Shell Programming and Scripting
Hi Experts,
Help needed on joining one line above & below to the pattern matched string line.
The input file, required output is mentioned below
Input file
ABCD DEFG5 42.0.1-63.38.31
KKKK iokl IP Connection Available
ABCD DEFG5 42.0.1-63.38.31
... (7 Replies)
Discussion started by: krao
7 Replies
7. Shell Programming and Scripting
Hi,
I have a file whose lines are something like
Tchampionspsq^@~^@^^^A^@^@^@^A^A^Aÿð^@^@^@^@^@^@^@^@^@^@^A^@^@^@^@^?ð^@^@^@^@^@^@^@?ð^@^@^@^@^@^@pppsq^@~^@#@^@^@^@^@^@^Hw^H^@^@^@^K^@^@^@^@xp^At^@^FTtime2psq^@ ~^@^^^A^@^@^@^B^A
I need to extract all words matching T*psq from the file.
Thing is... (4 Replies)
Discussion started by: shekhar2010us
4 Replies
8. Shell Programming and Scripting
I have a log file as given below
012/01/21 10:29:02 (111111) Processing Job '23_369468343464564'
2012/01/21 10:29:02 (111111)
Making Job '23_369468343464564.0'...
2012/01/21 10:29:04 (111111)
Jobnumber '23_369468343464564' was successful
2012/01/21 10:29:04 ... (12 Replies)
Discussion started by: rpm120
12 Replies
9. UNIX for Dummies Questions & Answers
Hi,
I would like to comment a line that matched a string "sreenivas" in a file without opening it.
Thanks in advance.
Regards,
Sreenivas (3 Replies)
Discussion started by: raosr020
3 Replies
10. Shell Programming and Scripting
i am having file like this
#!/bin/bash
read -p 'Username: ' uservar
match='<color="red" />'
text='this is only a test
so please be patient
<color="red" />'
echo "$text" | sed "s/$match/&$uservar\g"
so desireble output what i want is if user type MARIA
this is only a test
so please... (13 Replies)
Discussion started by: tomislav91
13 Replies
LEARN ABOUT DEBIAN
mergelogs
MERGELOGS(1) General Commands Manual MERGELOGS(1) NAME
mergelogs - merge and consolidate web server logs SYNOPSIS
mergelogs -p penlog [-c] [-d] [-j jitter] [-t seconds] server1:logfile1 [server2:logfile2 ...] EXAMPLES
mergelogs -p pen.log 10.0.0.1:access_log.1 10.0.0.2:access_log.2 mergelogs -p pen.log 10.0.18.6:access_log-10.0.18.6 10.0.18.8:access_log-10.0.18.8 DESCRIPTION
When pen is used to load balance web servers, the web server log file lists all accesses as coming from the host running pen. This makes it more difficult to analyze the log file. To solve this, pen creates its own log file, which contains the real client address, the time of the access, the target server address and the first few bytes of the requests. Mergelogs reads pen's log file and the log files of all load balanced web servers, compares each entry and creates a combined log file that looks as if the web server cluster were a single physical server. Client addresses are replaced with the real client addresses. In the event that no matching client address can be found in the pen log, the server address is used instead. This should never happen, and is meant as a debugging tool. A large number of these indicates that the server system date needs to be set, or that the jitter value is too small. You probably don't want to use this program. Penlog is a much more elegant and functional solution. OPTIONS
-c Do not cache pen log entries. The use of this option is not recommended, as it will make mergelogs search the entire pen log for every line in the web server logs. -d Debugging (repeat for more). -p penlog Log file from pen. -j jitter Jitter in seconds (default 600). This is the maximum variation in time stamps in the pen and web server log files. A smaller value will result in a smaller pen log cache and faster processing, at the risk of missed entries. -t seconds The difference in seconds between the time on the pen server and UTC. For example, this is 7200 (two hours) in Finland. server:logfile Web server address and name of log file. AUTHOR
Copyright (C) 2001-2003 Ulric Eriksson, <ulric@siag.nu>. SEE ALSO
pen(1), webresolve(1), penlog(1), penlogd(1) LOCAL MERGELOGS(1)