Script to extract line from logfile


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Script to extract line from logfile
# 8  
Old 09-20-2009
Hi jp2542a,

Thankyou so much dear,anything you ask.

The execution environment is windows.

Thanks again,
Garry
# 9  
Old 09-20-2009
I will do it... a couple of caveats/questions:
  1. Are you using the cgywin/bash envrionment?
  2. Are you willing to test the script and provide detailed feedback?
  3. What is the maximum airspeed of an unladen swallow? <joking>
# 10  
Old 09-21-2009
Hi jp2542a,

It is Bash environment.
And why not I will provide you with the feedback.

Thanks,
Garry
# 11  
Old 09-22-2009
I've restructured the awk script based on your request. We need to do a few things first.

First thing is to set up a test environment. Make a directory structure with files that starts with C:\gtest instead of C:\webapps. It should contain copies of a few of the webapps directory.

Then copy the following awk code into a file named log.awk. I've tried to add comments to help you understand what it is doing.

Code:
# log.awk - strip lines out of file and make log of lines removed

# This clause is executed on the opening of each file on the command line
# It first checks to see if this is not the first file and cleans up the
#  previous files and copies the new stripped file to the old file
#  NOTE:  I would prefer if it didn't overwrite the input file...
# It then creates the names and commands it will need later
(FNR == 1) {                    # check if first line of new file
        if ( NR != 1)           # see if this is not first file
        {
                close(FILEOUT)  # if true, then we close previous files
                close(FILELOG)
                system(FILECP)  # do the copy
                system(FILERM)  # remove the work file
        }

        FILEOUT = FILENAME ".strip"     # create striped file name using input file name
        FILELOG = FILENAME              # create test.txt path
        sub(/logfile_systemdat.log/, "test.txt", FILELOG)

        FILECP = "cp " FILEOUT " " FILENAME     # copy command
        FILERM = "rm " FILEOUT          # remove strip file
}

# This clause executes for every line
# It copies lines from the input file to the appropriate output file
{
        if ( $0 !~ /GuestUser/ )      # test for pattern in line
                print > FILEOUT # save to stripped file if pattern not found
        else
                print > FILELOG # save to remove log if pattern match
}

# This clause is executed when the last line of the last file is reached
END {
        system(FILECP)  # clean up last file
        system(FILERM)
}


Next execute the following command in the directory you put log.awk

Code:
awk -f log.awk C:\gtest\data\*\logs\logfile_systemdat.log

The * part should be a pattern that matches just the server names if there is something other than server names at that level.

Tell me if there are any error messages. Show me what you did and what you saw by copying the output screen. Check and see if you got what you expected in the test directories.

Once this works we will create a bash script that can be used by cron.
# 12  
Old 09-27-2009
Hi jp2542a,

Sorry for the late reply.I was travelling.

I copied the script into a notepad file and saved it as log.awk
And then copied some logfiles to this path - c\gtest\servername\logfiles
But did not understand how to execute the - awk -f log.awk C:\gtest\data\*\logs\logfile_systemdat.log

saved this line in a .bat file and tried to execute.
Let me reiterate the script has to be executed in windows environment.
And I think I made a mistake by replying bash enviroment,sorry for it.
Please guide me.

Thanks,
Garry
# 13  
Old 09-27-2009
geeze.. I thought this was a Unix and Linux Forum ;)

Garry,

You gonna have to meet me half way here. While my main battle computer and fly-with-me laptop are windows based, I (and I suspect most here) am a Solaris/Linux/Unix/C/shell/awk/kernel kinda guy. Yeah, I do Windows... Just don't like to admit it... Most of the solutions here are targeted toward Unix kind of environments....

So, you need to install cygwin to get a proper Unix/Linux environment on Windows. Google it Smilie. There are instructions on how to download and install it. If you need me to help you install it, I'm gonna charge you Smilie .

Once you get it installed, then a bash shell (looks sorta like a Windows cmd prompt window). It will be in your start menu under cygwin.You will have a Unix/Linux/Solaris type shell environment that includes the awk command.

Simply type the awk command to the bash prompt.
Smilie
# 14  
Old 09-27-2009
Or you can try Gawk for Windows Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Logfile monitoring with logfile replacement

Bonjour, I've wrote a script to monitor a logfile in realtime. It is working almost perfeclty except for two things. The script use the following technique : tail -fn0 $logfile | \ while read line ; do ... some stuff done First one, I'd like a way to end the monitoring script if a... (3 Replies)
Discussion started by: Warluck
3 Replies

2. Shell Programming and Scripting

Extract the last 10 minutes from logfile

Any better way to extract the last 10 minutes from logfile? Dec 18 09:41:18 aaa Dec 18 09:46:29 aa Dec 18 09:48:39 vvv Dec 18 09:48:54 bbb Dec 18 09:54:47 bbb Dec 18 09:55:33 fcf Dec 18 09:55:38 ssdf Dec 18 09:57:58 sdsds Dec 18 09:58:10 sdsd Dec 18 10:00:50 sdsd Dec 18... (5 Replies)
Discussion started by: timmywong
5 Replies

3. Shell Programming and Scripting

Unique line from logfile

Hi Team, Pls help me in fixing this problem. I have file called ORA_FILE.log Please find the log in that below: ************************ ORA-04061: existing state of package body "MST.ONE_PACKAGE" has been invalidated ORA-04065: not executed, altered or dropped package body... (2 Replies)
Discussion started by: indira_s
2 Replies

4. Shell Programming and Scripting

Extract three substrings from a logfile

I have a log file like below. 66.249.73.11 - - "UCiZ7QocVqYAABgwfP8AAHAA" "US" "Mediapartners-Google" "-" www.mahashwetha.com.sg "GET... (2 Replies)
Discussion started by: Tuxidow
2 Replies

5. Shell Programming and Scripting

HELP: Shell Script to read a Log file line by line and extract Info based on KEYWORDS matching

I have a LOG file which looks like this Import started at: Mon Jul 23 02:13:01 EDT 2012 Initialization completed in 2.146 seconds. -------------------------------------------------------------------------------- -- Import summary for Import item: PolicyInformation... (8 Replies)
Discussion started by: biztank
8 Replies

6. Shell Programming and Scripting

Search for a pattern,extract value(s) from next line, extract lines having those extracted value(s)

I have hundreds of files to process. In each file I need to look for a pattern then extract value(s) from next line and then search for value(s) selected from point (2) in the same file at a specific position. HEADER ELECTRON TRANSPORT 18-MAR-98 1A7V TITLE CYTOCHROME... (7 Replies)
Discussion started by: AshwaniSharma09
7 Replies

7. Shell Programming and Scripting

get the fifth line of a text file into a shell script and trim the line to extract a WORD

FOLKS , i have a text file that is generated automatically of an another korn shell script, i want to bring in the fifth line of the text file in to my korn shell script and look for a particular word in the line . Can you all share some thoughts on this one. thanks... Venu (3 Replies)
Discussion started by: venu
3 Replies

8. Shell Programming and Scripting

Logfile - extracting certain lines to concatenate into 1 line

I've got a log file from automatic diagnostic runs. The log file is appended to each time an automatic log is run. I'd like to just pull certain lines from each run in the log file, and concatenate them into 1 comma delimited line (for export into excel or an html table). Each diagnostic run... (3 Replies)
Discussion started by: BecTech
3 Replies

9. Shell Programming and Scripting

Need to extract specific pattern from logfile

Log File: Attempting to contact (DESCRIPTION=(SOURCE_ROUTE=OFF)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=hostname1.com)(PORT=1521)))(CONNECT_DATA=(SID=database1)(SRVR=DEDICATED))) Attempting to contact... (2 Replies)
Discussion started by: techychap
2 Replies

10. UNIX for Advanced & Expert Users

Script to Extract the line from File with specified offset

Hi All, I need to extract only XML details from large log file which may contain other unwanted junk details. For example, our xml will be start as <OUTBOUND_MESSAGE .....> and ends with </OUTBOUND_MESSAGE>. I want to extract only lines between these start and end tag (Including these tags)... (5 Replies)
Discussion started by: thinakarmani
5 Replies
Login or Register to Ask a Question