Visit Our UNIX and Linux User Community


Script to extract line from logfile


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to extract line from logfile
# 1  
Old 09-18-2009
Script to extract line from logfile

Hi ,
Can someone help me,I am not well versed with scripting,it is very urjent.

Need a script in perl or shell for the following task.

The logfile contains several entries of which one particular string to be searched and that complete line has to be removed and copied to another file to maitain a record.

The path of logfile "C:\webapps\data\servername\logs\logfile_systemdat.log

The string to be searched is "GuestUser"

The complete line is as follows.

10.100.102.5 - - [10-Sep-2009:00:01:59 -0700] 0 "GET /bsca/ APPS/1.1" 302 2 "-" "TMC Transaction Management Component Response Time Central/7.2.0+Action+GuestUser"

All the lines which contain the string Guestuser, has to be removed from the logfile and copied to another file to maintain a record.

The log files are generated on windows and Unix box - they are then copeid to windows box where the script has to run.

Thanks In Advance,
garry
# 2  
Old 09-18-2009
Try this,

Code:
sed -i -e '/GuestUser/w output_file' -e '/GuestUser/d'  input_file

Regards,

Ranjith
# 3  
Old 09-18-2009
Script to extract line from logfile

Hi Ranjith,

Appreciate your quick responce.
Can I get the complete script from opening the file from the path to reading the log.
Cut the line from logfile and save the file.

I also want to make a note here.
There would be several entries in the logfile.
Some start with date and some start with ip and the remaining part of the line differs with the login.
The sed you mentioned will that remove the complete line.

Thanks alot,
Garry
# 4  
Old 09-18-2009
awk is your friend :D

This is how you do it using awk.

This will create a files based on the input file names. The file without the lines you want removed will be <filename>.strip . The file with the lines removed will be <filename>.sav . <filename> is the path to the input file.

First make a file containing the awk code. I used log.awk for the example.

Code:
# strip lines out of file and make log of lines removed
# clause to make file name for output
(FNR == 1) {                    # check if first line of new file
        if ( NR != 1)           # see if this is not first file
        {
                close(FILEOUT)  # if true, then we close previous files
                close(FILELOG)
        }
        FILEOUT = FILENAME ".strip"     # create striped file name using input file name
        FILELOG = FILENAME ".sav"       # create file name for removed lines
}

# clause for ever line
{
        if ( $0 !~ /ntp/ )      # replace ntp with your pattern
                print > FILEOUT # save to stripped file if pattern not found
        else
                print > FILELOG # save to remove log if pattern match
}

Change the pattern to whatever you want to use. (GuestUser?)

Then execute it using:

Code:
awk -f log.awk <list of files>

Where <list of files> is a bunch of files you want to process. You get a set of .sav and .strip files in the same directory as the source file.

Note: I don't believe in modifying source files when not necessary. Tools such as this should not modify input files.

If for some reason you want to do the same thing in a single line, use this format:

Code:
awk '(FNR == 1) { if ( NR != 1) { close(FILEOUT); close(FILELOG) } FILEOUT = FILENAME ".strip"; FILELOG = FILENAME ".sav" } { if ( $0 !~ /YOUR-PATTERN-HERE/ ) print > FILEOUT; else print > FILELOG }' <list-of-files>

# 5  
Old 09-19-2009
Dear jp2542a,

Can you complete the script from line one for me.I understand what you write but I wont be able to complete the script as I do not have scripting knowledge.You will be a great help to me.

1.The source logfile has to be edited and all stripped lines should be copied to one file.
2. The path of the logfiles will be C:\webapps\data\servername\logs\logfile_systemdat.log
3. The script has to run everyday,based on system date, read thru the file for each server,strip the line - copy the line to test.txt and save the log file.

Thanks Alot,
Garry
# 6  
Old 09-19-2009
I normally charge for when asked to produce a production ready product Smilie.

What is your execution environment? (linux, solaris, windows ?)

---------- Post updated at 07:15 PM ---------- Previous update was at 07:14 PM ----------

I normally charge for when asked to produce a production ready product Smilie.

What is your execution environment? (linux, solaris, windows ?)
# 7  
Old 09-20-2009
Here's one way to do it with Perl:

Code:
@ARGV = ("f1");
$^I = ".bak";
open(OP, ">f1.out");
while (<>) {
  if (/xyz/) {print OP}
  else {print}
}

tyler_durden

Previous Thread | Next Thread
Test Your Knowledge in Computers #385
Difficulty: Easy
The Linux uptime command show how long the hard drive has been powered down.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Logfile monitoring with logfile replacement

Bonjour, I've wrote a script to monitor a logfile in realtime. It is working almost perfeclty except for two things. The script use the following technique : tail -fn0 $logfile | \ while read line ; do ... some stuff done First one, I'd like a way to end the monitoring script if a... (3 Replies)
Discussion started by: Warluck
3 Replies

2. Shell Programming and Scripting

Extract the last 10 minutes from logfile

Any better way to extract the last 10 minutes from logfile? Dec 18 09:41:18 aaa Dec 18 09:46:29 aa Dec 18 09:48:39 vvv Dec 18 09:48:54 bbb Dec 18 09:54:47 bbb Dec 18 09:55:33 fcf Dec 18 09:55:38 ssdf Dec 18 09:57:58 sdsds Dec 18 09:58:10 sdsd Dec 18 10:00:50 sdsd Dec 18... (5 Replies)
Discussion started by: timmywong
5 Replies

3. Shell Programming and Scripting

Unique line from logfile

Hi Team, Pls help me in fixing this problem. I have file called ORA_FILE.log Please find the log in that below: ************************ ORA-04061: existing state of package body "MST.ONE_PACKAGE" has been invalidated ORA-04065: not executed, altered or dropped package body... (2 Replies)
Discussion started by: indira_s
2 Replies

4. Shell Programming and Scripting

Extract three substrings from a logfile

I have a log file like below. 66.249.73.11 - - "UCiZ7QocVqYAABgwfP8AAHAA" "US" "Mediapartners-Google" "-" www.mahashwetha.com.sg "GET... (2 Replies)
Discussion started by: Tuxidow
2 Replies

5. Shell Programming and Scripting

HELP: Shell Script to read a Log file line by line and extract Info based on KEYWORDS matching

I have a LOG file which looks like this Import started at: Mon Jul 23 02:13:01 EDT 2012 Initialization completed in 2.146 seconds. -------------------------------------------------------------------------------- -- Import summary for Import item: PolicyInformation... (8 Replies)
Discussion started by: biztank
8 Replies

6. Shell Programming and Scripting

Search for a pattern,extract value(s) from next line, extract lines having those extracted value(s)

I have hundreds of files to process. In each file I need to look for a pattern then extract value(s) from next line and then search for value(s) selected from point (2) in the same file at a specific position. HEADER ELECTRON TRANSPORT 18-MAR-98 1A7V TITLE CYTOCHROME... (7 Replies)
Discussion started by: AshwaniSharma09
7 Replies

7. Shell Programming and Scripting

get the fifth line of a text file into a shell script and trim the line to extract a WORD

FOLKS , i have a text file that is generated automatically of an another korn shell script, i want to bring in the fifth line of the text file in to my korn shell script and look for a particular word in the line . Can you all share some thoughts on this one. thanks... Venu (3 Replies)
Discussion started by: venu
3 Replies

8. Shell Programming and Scripting

Logfile - extracting certain lines to concatenate into 1 line

I've got a log file from automatic diagnostic runs. The log file is appended to each time an automatic log is run. I'd like to just pull certain lines from each run in the log file, and concatenate them into 1 comma delimited line (for export into excel or an html table). Each diagnostic run... (3 Replies)
Discussion started by: BecTech
3 Replies

9. Shell Programming and Scripting

Need to extract specific pattern from logfile

Log File: Attempting to contact (DESCRIPTION=(SOURCE_ROUTE=OFF)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=hostname1.com)(PORT=1521)))(CONNECT_DATA=(SID=database1)(SRVR=DEDICATED))) Attempting to contact... (2 Replies)
Discussion started by: techychap
2 Replies

10. UNIX for Advanced & Expert Users

Script to Extract the line from File with specified offset

Hi All, I need to extract only XML details from large log file which may contain other unwanted junk details. For example, our xml will be start as <OUTBOUND_MESSAGE .....> and ends with </OUTBOUND_MESSAGE>. I want to extract only lines between these start and end tag (Including these tags)... (5 Replies)
Discussion started by: thinakarmani
5 Replies

Featured Tech Videos