03-22-2009
save every line in log file with matched string
i have been doing this script to match every line in a current log file (access_log) with strings that i list from a path (consist of 100 of user's name ex: meggae )..
and then make a directory of every string from the text file (/path/meggae/) --->if it matched..
then print every line from the log file if it matched the string into a new file name base on the string (meggae.txt) in the new created directory (/path/meggae/meggae.txt)..
below is the example script that i made..it has no error so far but it uses 100% of my processes as well as my memory and sometimes it uses my swap space and my lappy go lagged and hang..i hope some explanation and some solution can help me not to crash my dear lappy
Quote:
#!/usr/bin/perl -w
use File::Basename;
use CGI;
use CGI::Carp qw(fatalsToBrowser);
use File::Path;
use File::Copy;
print "Content-type: text/html\r\n";
print "\r\n";
open (example7, "/path/foo/foo/access_log") || die ("Could not open file. <br> $!");
@cisco = <example7>;
$new = qx(ls /path/foo/) or die "Could not read file. <br> $!"; #contain user's name ex: meggae and other 100 users
@passwd = split(' ',$new); #to split every string from <space> char
print $passwd; # try to see if it works and it worked well
foreach $string (sort(@passwd)) {
my $path = "/path/foo/foo/$string/log";
if (! -d $path)
{
mkpath($path,755) or die "Failed to create $path: $!\n";
}
foreach $line (sort(@cisco))
{
if ($line =~ $string)
{
foreach ($line =~ $string)
{
$file = "$string.txt";
$append = 0;
if ($append){
open (OUTFILE,">$file");
}else{
open (OUTFILE,">>$file");
}
foreach $line (@cisco)
{
foreach ($line =~ $string)
{print OUTFILE $line;}
}
close (OUTFILE);
}
foreach ($path =~ $string) {
move($file,$path) or die "Failed to copy $file: $!\n"; }
}
}
}
any solutions will tremendously help me with this space/memory crashing and what have i done wrong
10 More Discussions You Might Find Interesting
1. UNIX for Advanced & Expert Users
All,
Is there any way out to display the nth line before the string is matched ???
Eg : If i have a file which has the following contents and if i want to get the
3rd line before the string is matched
a
b
c
d
e
f
if i give the input as f and lines before the match as 3 then it should... (5 Replies)
Discussion started by: helper
5 Replies
2. Shell Programming and Scripting
Hi
I want to read a file line by line and search for a particular string in each line(say for example string containing @ )and save that string into a variable.
Can someone suggest me the way to implement it.I am using K- shell
Thanks
Ishita (5 Replies)
Discussion started by: Ishita
5 Replies
3. Shell Programming and Scripting
Hi ,
I have input file and i want to extract below strings
<msisdn xmlns="">0492001956</ msisdn> => numaber inside brackets
<resCode>3000</resCode> => 3000 needs to be extracted
<resMessage>Request time
getBalances_PSM.c(37): d out</resMessage></ns2:getBalancesResponse> => the word... (14 Replies)
Discussion started by: sushmab82
14 Replies
4. UNIX for Advanced & Expert Users
Hi,
I have a query as follows :
suppose I am matching a string in a file say "start from here"
and I want to pick up 'n' number of lines () from the matched string. Is there any way to do that ?
1) going forward I want to do this for every match for the above string
2) or limit this to... (2 Replies)
Discussion started by: sumoka
2 Replies
5. Shell Programming and Scripting
Hi Experts,
I am very new to scripting and have a prb since few days and it is urgent to solve so much appreciated if u help me.
i have 2 files
file1.txt
9647810043118
9647810043126
9647810043155
9647810043161
9647810043166
9647810043185
9647810043200
9647810043203
9647810043250... (22 Replies)
Discussion started by: mustafa.abdulsa
22 Replies
6. Shell Programming and Scripting
Hi Experts,
Help needed on joining one line above & below to the pattern matched string line.
The input file, required output is mentioned below
Input file
ABCD DEFG5 42.0.1-63.38.31
KKKK iokl IP Connection Available
ABCD DEFG5 42.0.1-63.38.31
... (7 Replies)
Discussion started by: krao
7 Replies
7. Shell Programming and Scripting
Hi,
I have a file whose lines are something like
Tchampionspsq^@~^@^^^A^@^@^@^A^A^Aÿð^@^@^@^@^@^@^@^@^@^@^A^@^@^@^@^?ð^@^@^@^@^@^@^@?ð^@^@^@^@^@^@pppsq^@~^@#@^@^@^@^@^@^Hw^H^@^@^@^K^@^@^@^@xp^At^@^FTtime2psq^@ ~^@^^^A^@^@^@^B^A
I need to extract all words matching T*psq from the file.
Thing is... (4 Replies)
Discussion started by: shekhar2010us
4 Replies
8. Shell Programming and Scripting
I have a log file as given below
012/01/21 10:29:02 (111111) Processing Job '23_369468343464564'
2012/01/21 10:29:02 (111111)
Making Job '23_369468343464564.0'...
2012/01/21 10:29:04 (111111)
Jobnumber '23_369468343464564' was successful
2012/01/21 10:29:04 ... (12 Replies)
Discussion started by: rpm120
12 Replies
9. UNIX for Dummies Questions & Answers
Hi,
I would like to comment a line that matched a string "sreenivas" in a file without opening it.
Thanks in advance.
Regards,
Sreenivas (3 Replies)
Discussion started by: raosr020
3 Replies
10. Shell Programming and Scripting
i am having file like this
#!/bin/bash
read -p 'Username: ' uservar
match='<color="red" />'
text='this is only a test
so please be patient
<color="red" />'
echo "$text" | sed "s/$match/&$uservar\g"
so desireble output what i want is if user type MARIA
this is only a test
so please... (13 Replies)
Discussion started by: tomislav91
13 Replies
LEARN ABOUT MOJAVE
file::basename5.18
File::Basename(3pm) Perl Programmers Reference Guide File::Basename(3pm)
NAME
File::Basename - Parse file paths into directory, filename and suffix.
SYNOPSIS
use File::Basename;
($name,$path,$suffix) = fileparse($fullname,@suffixlist);
$name = fileparse($fullname,@suffixlist);
$basename = basename($fullname,@suffixlist);
$dirname = dirname($fullname);
DESCRIPTION
These routines allow you to parse file paths into their directory, filename and suffix.
NOTE: "dirname()" and "basename()" emulate the behaviours, and quirks, of the shell and C functions of the same name. See each function's
documentation for details. If your concern is just parsing paths it is safer to use File::Spec's "splitpath()" and "splitdir()" methods.
It is guaranteed that
# Where $path_separator is / for Unix, for Windows, etc...
dirname($path) . $path_separator . basename($path);
is equivalent to the original path for all systems but VMS.
"fileparse"
my($filename, $directories, $suffix) = fileparse($path);
my($filename, $directories, $suffix) = fileparse($path, @suffixes);
my $filename = fileparse($path, @suffixes);
The "fileparse()" routine divides a file path into its $directories, $filename and (optionally) the filename $suffix.
$directories contains everything up to and including the last directory separator in the $path including the volume (if applicable).
The remainder of the $path is the $filename.
# On Unix returns ("baz", "/foo/bar/", "")
fileparse("/foo/bar/baz");
# On Windows returns ("baz", 'C:fooar', "")
fileparse('C:fooaraz');
# On Unix returns ("", "/foo/bar/baz/", "")
fileparse("/foo/bar/baz/");
If @suffixes are given each element is a pattern (either a string or a "qr//") matched against the end of the $filename. The matching
portion is removed and becomes the $suffix.
# On Unix returns ("baz", "/foo/bar/", ".txt")
fileparse("/foo/bar/baz.txt", qr/.[^.]*/);
If type is non-Unix (see "fileparse_set_fstype") then the pattern matching for suffix removal is performed case-insensitively, since
those systems are not case-sensitive when opening existing files.
You are guaranteed that "$directories . $filename . $suffix" will denote the same location as the original $path.
"basename"
my $filename = basename($path);
my $filename = basename($path, @suffixes);
This function is provided for compatibility with the Unix shell command basename(1). It does NOT always return the file name portion
of a path as you might expect. To be safe, if you want the file name portion of a path use "fileparse()".
"basename()" returns the last level of a filepath even if the last level is clearly directory. In effect, it is acting like "pop()"
for paths. This differs from "fileparse()"'s behaviour.
# Both return "bar"
basename("/foo/bar");
basename("/foo/bar/");
@suffixes work as in "fileparse()" except all regex metacharacters are quoted.
# These two function calls are equivalent.
my $filename = basename("/foo/bar/baz.txt", ".txt");
my $filename = fileparse("/foo/bar/baz.txt", qr/Q.txtE/);
Also note that in order to be compatible with the shell command, "basename()" does not strip off a suffix if it is identical to the
remaining characters in the filename.
"dirname"
This function is provided for compatibility with the Unix shell command dirname(1) and has inherited some of its quirks. In spite of
its name it does NOT always return the directory name as you might expect. To be safe, if you want the directory name of a path use
"fileparse()".
Only on VMS (where there is no ambiguity between the file and directory portions of a path) and AmigaOS (possibly due to an
implementation quirk in this module) does "dirname()" work like "fileparse($path)", returning just the $directories.
# On VMS and AmigaOS
my $directories = dirname($path);
When using Unix or MSDOS syntax this emulates the dirname(1) shell function which is subtly different from how "fileparse()" works. It
returns all but the last level of a file path even if the last level is clearly a directory. In effect, it is not returning the
directory portion but simply the path one level up acting like "chop()" for file paths.
Also unlike "fileparse()", "dirname()" does not include a trailing slash on its returned path.
# returns /foo/bar. fileparse() would return /foo/bar/
dirname("/foo/bar/baz");
# also returns /foo/bar despite the fact that baz is clearly a
# directory. fileparse() would return /foo/bar/baz/
dirname("/foo/bar/baz/");
# returns '.'. fileparse() would return 'foo/'
dirname("foo/");
Under VMS, if there is no directory information in the $path, then the current default device and directory is used.
"fileparse_set_fstype"
my $type = fileparse_set_fstype();
my $previous_type = fileparse_set_fstype($type);
Normally File::Basename will assume a file path type native to your current operating system (ie. /foo/bar style on Unix, fooar on
Windows, etc...). With this function you can override that assumption.
Valid $types are "MacOS", "VMS", "AmigaOS", "OS2", "RISCOS", "MSWin32", "DOS" (also "MSDOS" for backwards bug compatibility), "Epoc"
and "Unix" (all case-insensitive). If an unrecognized $type is given "Unix" will be assumed.
If you've selected VMS syntax, and the file specification you pass to one of these routines contains a "/", they assume you are using
Unix emulation and apply the Unix syntax rules instead, for that function call only.
SEE ALSO
dirname(1), basename(1), File::Spec
perl v5.18.2 2014-01-06 File::Basename(3pm)