|
|
Sponsored Content
Top Forums
Shell Programming and Scripting
save every line in log file with matched string
Post 302299921 by meggae on Sunday 22nd of March 2009 08:24:27 AM
03-22-2009
save every line in log file with matched string
i have been doing this script to match every line in a current log file (access_log) with strings that i list from a path (consist of 100 of user's name ex: meggae )..
and then make a directory of every string from the text file (/path/meggae/) --->if it matched..
then print every line from the log file if it matched the string into a new file name base on the string (meggae.txt) in the new created directory (/path/meggae/meggae.txt)..
below is the example script that i made..it has no error so far but it uses 100% of my processes as well as my memory and sometimes it uses my swap space and my lappy go lagged and hang..i hope some explanation and some solution can help me not to crash my dear lappy
any solutions will tremendously help me with this space/memory crashing and what have i done wrong
and then make a directory of every string from the text file (/path/meggae/) --->if it matched..
then print every line from the log file if it matched the string into a new file name base on the string (meggae.txt) in the new created directory (/path/meggae/meggae.txt)..
below is the example script that i made..it has no error so far but it uses 100% of my processes as well as my memory and sometimes it uses my swap space and my lappy go lagged and hang..i hope some explanation and some solution can help me not to crash my dear lappy
Quote:
#!/usr/bin/perl -w
use File::Basename;
use CGI;
use CGI::Carp qw(fatalsToBrowser);
use File::Path;
use File::Copy;
print "Content-type: text/html\r\n";
print "\r\n";
open (example7, "/path/foo/foo/access_log") || die ("Could not open file. <br> $!");
@cisco = <example7>;
$new = qx(ls /path/foo/) or die "Could not read file. <br> $!"; #contain user's name ex: meggae and other 100 users
@passwd = split(' ',$new); #to split every string from <space> char
print $passwd; # try to see if it works and it worked well
foreach $string (sort(@passwd)) {
my $path = "/path/foo/foo/$string/log";
if (! -d $path)
{
mkpath($path,755) or die "Failed to create $path: $!\n";
}
foreach $line (sort(@cisco))
{
if ($line =~ $string)
{
foreach ($line =~ $string)
{
$file = "$string.txt";
$append = 0;
if ($append){
open (OUTFILE,">$file");
}else{
open (OUTFILE,">>$file");
}
foreach $line (@cisco)
{
foreach ($line =~ $string)
{print OUTFILE $line;}
}
close (OUTFILE);
}
foreach ($path =~ $string) {
move($file,$path) or die "Failed to copy $file: $!\n"; }
}
}
}
use File::Basename;
use CGI;
use CGI::Carp qw(fatalsToBrowser);
use File::Path;
use File::Copy;
print "Content-type: text/html\r\n";
print "\r\n";
open (example7, "/path/foo/foo/access_log") || die ("Could not open file. <br> $!");
@cisco = <example7>;
$new = qx(ls /path/foo/) or die "Could not read file. <br> $!"; #contain user's name ex: meggae and other 100 users
@passwd = split(' ',$new); #to split every string from <space> char
print $passwd; # try to see if it works and it worked well
foreach $string (sort(@passwd)) {
my $path = "/path/foo/foo/$string/log";
if (! -d $path)
{
mkpath($path,755) or die "Failed to create $path: $!\n";
}
foreach $line (sort(@cisco))
{
if ($line =~ $string)
{
foreach ($line =~ $string)
{
$file = "$string.txt";
$append = 0;
if ($append){
open (OUTFILE,">$file");
}else{
open (OUTFILE,">>$file");
}
foreach $line (@cisco)
{
foreach ($line =~ $string)
{print OUTFILE $line;}
}
close (OUTFILE);
}
foreach ($path =~ $string) {
move($file,$path) or die "Failed to copy $file: $!\n"; }
}
}
}
|
|
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
All,
Is there any way out to display the nth line before the string is matched ???
Eg : If i have a file which has the following contents and if i want to get the
3rd line before the string is matched
a
b
c
d
e
f
if i give the input as f and lines before the match as 3 then it should... (5 Replies)
Discussion started by: helper
5 Replies
2. Shell Programming and Scripting
Hi
I want to read a file line by line and search for a particular string in each line(say for example string containing @ )and save that string into a variable.
Can someone suggest me the way to implement it.I am using K- shell
Thanks
Ishita (5 Replies)
Discussion started by: Ishita
5 Replies
3. Shell Programming and Scripting
Hi ,
I have input file and i want to extract below strings
<msisdn xmlns="">0492001956</ msisdn> => numaber inside brackets
<resCode>3000</resCode> => 3000 needs to be extracted
<resMessage>Request time
getBalances_PSM.c(37): d out</resMessage></ns2:getBalancesResponse> => the word... (14 Replies)
Discussion started by: sushmab82
14 Replies
4. UNIX for Advanced & Expert Users
Hi,
I have a query as follows :
suppose I am matching a string in a file say "start from here"
and I want to pick up 'n' number of lines () from the matched string. Is there any way to do that ?
1) going forward I want to do this for every match for the above string
2) or limit this to... (2 Replies)
Discussion started by: sumoka
2 Replies
5. Shell Programming and Scripting
Hi Experts,
I am very new to scripting and have a prb since few days and it is urgent to solve so much appreciated if u help me.
i have 2 files
file1.txt
9647810043118
9647810043126
9647810043155
9647810043161
9647810043166
9647810043185
9647810043200
9647810043203
9647810043250... (22 Replies)
Discussion started by: mustafa.abdulsa
22 Replies
6. Shell Programming and Scripting
Hi Experts,
Help needed on joining one line above & below to the pattern matched string line.
The input file, required output is mentioned below
Input file
ABCD DEFG5 42.0.1-63.38.31
KKKK iokl IP Connection Available
ABCD DEFG5 42.0.1-63.38.31
... (7 Replies)
Discussion started by: krao
7 Replies
7. Shell Programming and Scripting
Hi,
I have a file whose lines are something like
Tchampionspsq^@~^@^^^A^@^@^@^A^A^Aÿð^@^@^@^@^@^@^@^@^@^@^A^@^@^@^@^?ð^@^@^@^@^@^@^@?ð^@^@^@^@^@^@pppsq^@~^@#@^@^@^@^@^@^Hw^H^@^@^@^K^@^@^@^@xp^At^@^FTtime2psq^@ ~^@^^^A^@^@^@^B^A
I need to extract all words matching T*psq from the file.
Thing is... (4 Replies)
Discussion started by: shekhar2010us
4 Replies
8. Shell Programming and Scripting
I have a log file as given below
012/01/21 10:29:02 (111111) Processing Job '23_369468343464564'
2012/01/21 10:29:02 (111111)
Making Job '23_369468343464564.0'...
2012/01/21 10:29:04 (111111)
Jobnumber '23_369468343464564' was successful
2012/01/21 10:29:04 ... (12 Replies)
Discussion started by: rpm120
12 Replies
9. UNIX for Dummies Questions & Answers
Hi,
I would like to comment a line that matched a string "sreenivas" in a file without opening it.
Thanks in advance.
Regards,
Sreenivas (3 Replies)
Discussion started by: raosr020
3 Replies
10. Shell Programming and Scripting
i am having file like this
#!/bin/bash
read -p 'Username: ' uservar
match='<color="red" />'
text='this is only a test
so please be patient
<color="red" />'
echo "$text" | sed "s/$match/&$uservar\g"
so desireble output what i want is if user type MARIA
this is only a test
so please... (13 Replies)
Discussion started by: tomislav91
13 Replies
LEARN ABOUT DEBIAN
control.ctl
CONTROL.CTL(5) File Formats Manual CONTROL.CTL(5) NAME
control.ctl - specify handling of Usenet control messages DESCRIPTION
The file /etc/news/control.ctl is used to determine what action is taken when a control message is received. It is read by the parsecon- trol script, which is called by all the control scripts. (For an explanation of how the control scripts are invoked, see innd(8).) The file consists of a series of lines; blank lines and lines beginning with a number sign (``#'') are ignored. All other lines consist of four fields separated by a colon: message:from:newsgroups:action The first field is the name of the message for which this line is valid. It should be either the name of the control message, or the word ``all'' to mean that it is valid for all messages. The second field is a shell-style pattern that matches the email address of the person posting the message. (The poster's address is first converted to lowercase.) The matching is done using the shell's case statement; see sh (1) for details. If the control message is ``newgroup'' or ``rmgroup'' then the third field specifies the shell-style pattern that must match the group being created or removed. If the control message is of a different type, then this field is ignored. The fourth field specifies what action to take if this line is selected for the message. The following actions are understood: doit The action requested by the control message should be performed. In most cases the control script will also send mail to usenet. doifarg If the control message has an argument, this is treated as a ``doit'' action. If no argument was given, it is treated as a ``mail'' entry. This is used in ``sendsys'' entries script so that a site can request its own newsfeeds(5) entry by posting a ``sendsys mysite'' article. On the other hand, sendsys ``bombs'' ask that the entire newsfeeds file be sent to a forged reply-to address; by using ``doifarg'' such messages will not be processed automatically. doit=file The action is performed, but a log entry is written to the specified log file, file. If file is the word ``mail'' then the record is mailed. A null string is equivalent to /dev/null. A pathname that starts with a slash is taken as the absolute filename to use as the log. All other pathnames are written to /var/log/news/file.log. The log is written by writelog (see newslog(8)). drop No action is taken; the message is ignored. log A one-line log notice is sent to standard error. Innd normally directs this to the file /var/log/news/errlog. log=file A log entry is written to the specified log file, file, which is interpreted as described above. mail A mail message is sent to the news administrator. Lines are matched in order; the last match found in the file is the one that is used. For example, with the following three lines: newgroup:*:*:drop newgroup:tale@*.uu.net:comp.*|misc.*|news.*|rec.*|sci.*|soc.*|talk.*:doit newgroup:kre@munnari.oz.au:aus.*:mail A newgroup coming from ``tale'' at a UUNET machine will be honored if it is in the mainstream Usenet hierarchy. If ``kre'' posts a new- group message creating ``aus.foo'', then mail will be sent. All other newgroup messages are ignored. HISTORY
Written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews. This is revision 1.8, dated 1996/09/06. SEE ALSO
innd(8), newsfeeds(5), scanlogs(8). CONTROL.CTL(5)