Sponsored Content
Top Forums Shell Programming and Scripting save every line in log file with matched string Post 302299921 by meggae on Sunday 22nd of March 2009 08:24:27 AM
Old 03-22-2009
Question save every line in log file with matched string

i have been doing this script to match every line in a current log file (access_log) with strings that i list from a path (consist of 100 of user's name ex: meggae )..
and then make a directory of every string from the text file (/path/meggae/) --->if it matched..
then print every line from the log file if it matched the string into a new file name base on the string (meggae.txt) in the new created directory (/path/meggae/meggae.txt)..

below is the example script that i made..it has no error so far but it uses 100% of my processes as well as my memory and sometimes it uses my swap space and my lappy go lagged and hang..i hope some explanation and some solution can help me not to crash my dear lappy Smilie


Quote:
#!/usr/bin/perl -w
use File::Basename;
use CGI;
use CGI::Carp qw(fatalsToBrowser);
use File::Path;
use File::Copy;
print "Content-type: text/html\r\n";
print "\r\n";

open (example7, "/path/foo/foo/access_log") || die ("Could not open file. <br> $!");
@cisco = <example7>;

$new = qx(ls /path/foo/) or die "Could not read file. <br> $!"; #contain user's name ex: meggae and other 100 users

@passwd = split(' ',$new); #to split every string from <space> char
print $passwd; # try to see if it works and it worked well

foreach $string (sort(@passwd)) {

my $path = "/path/foo/foo/$string/log";

if (! -d $path)
{
mkpath($path,755) or die "Failed to create $path: $!\n";
}

foreach $line (sort(@cisco))
{

if ($line =~ $string)
{
foreach ($line =~ $string)
{
$file = "$string.txt";
$append = 0;
if ($append){
open (OUTFILE,">$file");
}else{
open (OUTFILE,">>$file");
}
foreach $line (@cisco)
{
foreach ($line =~ $string)
{print OUTFILE $line;}
}
close (OUTFILE);
}
foreach ($path =~ $string) {
move($file,$path) or die "Failed to copy $file: $!\n"; }

}

}
}
any solutions will tremendously help me with this space/memory crashing and what have i done wrong Smilie
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Command to display nth line before the string is matched.

All, Is there any way out to display the nth line before the string is matched ??? Eg : If i have a file which has the following contents and if i want to get the 3rd line before the string is matched a b c d e f if i give the input as f and lines before the match as 3 then it should... (5 Replies)
Discussion started by: helper
5 Replies

2. Shell Programming and Scripting

search a string in a line and save it in a variable

Hi I want to read a file line by line and search for a particular string in each line(say for example string containing @ )and save that string into a variable. Can someone suggest me the way to implement it.I am using K- shell Thanks Ishita (5 Replies)
Discussion started by: Ishita
5 Replies

3. Shell Programming and Scripting

Extracting particular string in a file and storing matched string in output file

Hi , I have input file and i want to extract below strings <msisdn xmlns="">0492001956</ msisdn> => numaber inside brackets <resCode>3000</resCode> => 3000 needs to be extracted <resMessage>Request time getBalances_PSM.c(37): d out</resMessage></ns2:getBalancesResponse> => the word... (14 Replies)
Discussion started by: sushmab82
14 Replies

4. UNIX for Advanced & Expert Users

capture data from matched string/line

Hi, I have a query as follows : suppose I am matching a string in a file say "start from here" and I want to pick up 'n' number of lines () from the matched string. Is there any way to do that ? 1) going forward I want to do this for every match for the above string 2) or limit this to... (2 Replies)
Discussion started by: sumoka
2 Replies

5. Shell Programming and Scripting

Match a line in File 1 with Column in File 2 and print whole line in file 2 when matched

Hi Experts, I am very new to scripting and have a prb since few days and it is urgent to solve so much appreciated if u help me. i have 2 files file1.txt 9647810043118 9647810043126 9647810043155 9647810043161 9647810043166 9647810043185 9647810043200 9647810043203 9647810043250... (22 Replies)
Discussion started by: mustafa.abdulsa
22 Replies

6. Shell Programming and Scripting

Help required on joining one line above & below to the pattern matched string line.

Hi Experts, Help needed on joining one line above & below to the pattern matched string line. The input file, required output is mentioned below Input file ABCD DEFG5 42.0.1-63.38.31 KKKK iokl IP Connection Available ABCD DEFG5 42.0.1-63.38.31 ... (7 Replies)
Discussion started by: krao
7 Replies

7. Shell Programming and Scripting

Print only matched string instead of entire line

Hi, I have a file whose lines are something like Tchampionspsq^@~^@^^^A^@^@^@^A^A^Aÿð^@^@^@^@^@^@^@^@^@^@^A^@^@^@^@^?ð^@^@^@^@^@^@^@?ð^@^@^@^@^@^@pppsq^@~^@#@^@^@^@^@^@^Hw^H^@^@^@^K^@^@^@^@xp^At^@^FTtime2psq^@ ~^@^^^A^@^@^@^B^A I need to extract all words matching T*psq from the file. Thing is... (4 Replies)
Discussion started by: shekhar2010us
4 Replies

8. Shell Programming and Scripting

ksh : need to get the 4 th line above and 2 nd below the matched pattern in the log file

I have a log file as given below 012/01/21 10:29:02 (111111) Processing Job '23_369468343464564' 2012/01/21 10:29:02 (111111) Making Job '23_369468343464564.0'... 2012/01/21 10:29:04 (111111) Jobnumber '23_369468343464564' was successful 2012/01/21 10:29:04 ... (12 Replies)
Discussion started by: rpm120
12 Replies

9. UNIX for Dummies Questions & Answers

Commenting a line matched with a specific string in a file

Hi, I would like to comment a line that matched a string "sreenivas" in a file without opening it. Thanks in advance. Regards, Sreenivas (3 Replies)
Discussion started by: raosr020
3 Replies

10. Shell Programming and Scripting

Insert a user input string after matched string in file

i am having file like this #!/bin/bash read -p 'Username: ' uservar match='<color="red" />' text='this is only a test so please be patient <color="red" />' echo "$text" | sed "s/$match/&$uservar\g" so desireble output what i want is if user type MARIA this is only a test so please... (13 Replies)
Discussion started by: tomislav91
13 Replies
Path::Class(3)						User Contributed Perl Documentation					    Path::Class(3)

NAME
Path::Class - Cross-platform path specification manipulation VERSION
version 0.33 SYNOPSIS
use Path::Class; my $dir = dir('foo', 'bar'); # Path::Class::Dir object my $file = file('bob', 'file.txt'); # Path::Class::File object # Stringifies to 'foo/bar' on Unix, 'fooar' on Windows, etc. print "dir: $dir "; # Stringifies to 'bob/file.txt' on Unix, 'bobfile.txt' on Windows print "file: $file "; my $subdir = $dir->subdir('baz'); # foo/bar/baz my $parent = $subdir->parent; # foo/bar my $parent2 = $parent->parent; # foo my $dir2 = $file->dir; # bob # Work with foreign paths use Path::Class qw(foreign_file foreign_dir); my $file = foreign_file('Mac', ':foo:file.txt'); print $file->dir; # :foo: print $file->as_foreign('Win32'); # foofile.txt # Interact with the underlying filesystem: # $dir_handle is an IO::Dir object my $dir_handle = $dir->open or die "Can't read $dir: $!"; # $file_handle is an IO::File object my $file_handle = $file->open($mode) or die "Can't read $file: $!"; DESCRIPTION
"Path::Class" is a module for manipulation of file and directory specifications (strings describing their locations, like '/home/ken/foo.txt' or 'C:WindowsFoo.txt') in a cross-platform manner. It supports pretty much every platform Perl runs on, including Unix, Windows, Mac, VMS, Epoc, Cygwin, OS/2, and NetWare. The well-known module File::Spec also provides this service, but it's sort of awkward to use well, so people sometimes avoid it, or use it in a way that won't actually work properly on platforms significantly different than the ones they've tested their code on. In fact, "Path::Class" uses "File::Spec" internally, wrapping all the unsightly details so you can concentrate on your application code. Whereas "File::Spec" provides functions for some common path manipulations, "Path::Class" provides an object-oriented model of the world of path specifications and their underlying semantics. "File::Spec" doesn't create any objects, and its classes represent the different ways in which paths must be manipulated on various platforms (not a very intuitive concept). "Path::Class" creates objects representing files and directories, and provides methods that relate them to each other. For instance, the following "File::Spec" code: my $absolute = File::Spec->file_name_is_absolute( File::Spec->catfile( @dirs, $file ) ); can be written using "Path::Class" as my $absolute = Path::Class::File->new( @dirs, $file )->is_absolute; or even as my $absolute = file( @dirs, $file )->is_absolute; Similar readability improvements should happen all over the place when using "Path::Class". Using "Path::Class" can help solve real problems in your code too - for instance, how many people actually take the "volume" (like "C:" on Windows) into account when writing "File::Spec"-using code? I thought not. But if you use "Path::Class", your file and directory objects will know what volumes they refer to and do the right thing. The guts of the "Path::Class" code live in the Path::Class::File and Path::Class::Dir modules, so please see those modules' documentation for more details about how to use them. EXPORT The following functions are exported by default. file A synonym for "Path::Class::File->new". dir A synonym for "Path::Class::Dir->new". If you would like to prevent their export, you may explicitly pass an empty list to perl's "use", i.e. "use Path::Class ()". The following are exported only on demand. foreign_file A synonym for "Path::Class::File->new_foreign". foreign_dir A synonym for "Path::Class::Dir->new_foreign". tempdir Create a new Path::Class::Dir instance pointed to temporary directory. my $temp = Path::Class::tempdir(CLEANUP => 1); A synonym for "Path::Class::Dir->new(File::Temp::tempdir(@_))". Notes on Cross-Platform Compatibility Although it is much easier to write cross-platform-friendly code with this module than with "File::Spec", there are still some issues to be aware of. o On some platforms, notably VMS and some older versions of DOS (I think), all filenames must have an extension. Thus if you create a file called foo/bar and then ask for a list of files in the directory foo, you may find a file called bar. instead of the bar you were expecting. Thus it might be a good idea to use an extension in the first place. AUTHOR
Ken Williams, KWILLIAMS@cpan.org COPYRIGHT
Copyright (c) Ken Williams. All rights reserved. This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. SEE ALSO
Path::Class::Dir, Path::Class::File, File::Spec perl v5.18.2 2017-10-06 Path::Class(3)
All times are GMT -4. The time now is 09:26 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy