Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Need help with unix command

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need help with unix command
# 1  
Old 12-30-2010
Need help with unix command
i have a requirement for searching iislogs(users logged in) for a specific duration for eg from 5:30AM to 7:30AM.

Please find below the logs pattern from IISLogs.
Code:
2010-12-22 05:00:48 10.253.129.252 sitescope@999999 172.24.64.49 443 GET /Jakarta/isapi_redirector2.dll SMTRANSACTIONID=1b4019ac-06a8-4d118600-0d50-013a0c52


The above line is one single line and i need to display lines which fall between starttime and endtime including starttimes and endtimes.

The options currenlty i tried are
1) Using SED for cutting the logs starting from one pattern to other pattern ( sed '/pattern1/pattern2/p' ). However im not sure if the logs contain the pattern that im searching for.. for example if i want to search for logs from 05:30 to 07:30 and there is no log in the source file for 05:30 my logic fails.
2) Second logic i tried is using sed and multiple awks for getting the logs by comparing if the log time falls in between starttime and end time defined in the command.
This logic works but it is very slow as i have to compare millions of lines.

Can i acheive this using awk command only once.I tried Fieldwidths, however awk is not able to split the logs according the field widths defined by me.

Can you please suggest some logic that is very fast. Also after the check is done for startime and endtime the command should return the complete line.


Last edited by DukeNuke2; 12-30-2010 at 07:10 AM..
# 2  
Old 12-30-2010
Quote:
Can i acheive this using awk command only once.
Should be possible.
Try to post a larger data sample and required output. Your request is not very clear.
# 3  
Old 12-30-2010
danmero,

please find below the input format.

2010-12-22 05:01:37 216.43.41.82 - 172.24.64.49 443 GET /images/blank_pixel.gif SMTRANSACTIONID=1b4019ac-06a8-4d118631-0d50-01b7156a 200 413 3137 0 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+InfoPath.2;+MS-RTC+LM+8)<EOL>
2010-12-22 05:01:37 216.43.41.82 - 172.24.64.49 443 GET /images/icon_preferences.gif SMTRANSACTIONID=1b4019ac-06a8-4d118631-07cc-02d00407 200 1447 3142 0 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+InfoPath.2;+MS-RTC+LM+8)<EOL>
2010-12-22 05:02:07 216.43.41.82 kroberson@icsco 172.24.64.49 443 GET /Jakarta/isapi_redirector2.dll SMTRANSACTIONID=1b4019ac-06a8-4d11864f-0c48-031f70f5 200 3351 3095 15 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+InfoPath.2;+MS-RTC+LM+8)
2010-12-22 05:04:36 216.43.41.82 kroberson@icsco 172.24.64.49 443 GET /Jakarta/isapi_redirector2.dll SMTRANSACTIONID=1b4019ac-06a8-4d1186e4-0b94-034027a3 200 1396 3138 0 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+InfoPath.2;+MS-RTC+LM+8)<EOL>
2010-12-22 05:05:37 216.43.41.82 kroberson@icsco 172.24.64.49 443 GET /Jakarta/isapi_redirector2.dll SMTRANSACTIONID=1b4019ac-06a8-4d118721-0b94-00fe2d34 200 1546 3138 0 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+InfoPath.2;+MS-RTC+LM+8)<EOL>

every line int the log starts with date and timestamp. in the following format yyyy-mm-dd HH:MM:SS.

command should search for log entries between 05:02 (HH:MM) and 05:04 (HH:MM) including the starttime and endtime.( start time and end time can be any time in HH:MM format)

Please find below the output ( lines )

2010-12-22 05:02:07 216.43.41.82 kroberson@icsco 172.24.64.49 443 GET /Jakarta/isapi_redirector2.dll SMTRANSACTIONID=1b4019ac-06a8-4d11864f-0c48-031f70f5 200 3351 3095 15 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+InfoPath.2;+MS-RTC+LM+8) <EOL>
2010-12-22 05:04:36 216.43.41.82 kroberson@icsco 172.24.64.49 443 GET /Jakarta/isapi_redirector2.dll SMTRANSACTIONID=1b4019ac-06a8-4d1186e4-0b94-034027a3 200 1396 3138 0 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+InfoPath.2;+MS-RTC+LM+8) <EOL>


Note: marked end of line in input and output format as <EOL>

let me know, if the example is not clear
# 4  
Old 12-30-2010
Something like this:

Code:
start_time="0502"
end_time="0504" 
awk -v va1=$start_time -v va2=$end_time '{p=substr($2,1,2)substr($2,4,2); if(p>=va1 && p<=va2) { print $0 } }' input_file

A further tuning based on your requirement is required ( check for date etc )
This User Gave Thanks to panyam For This Post:
saikiran_1984
# 5  
Old 12-30-2010
Thank you very much Panyam. it is working
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

UNIX command to copy files from Windows to UNIX box

Hi Folks, I have a file name abc.xml in my windows machine at the location c:\ytr\abc.xml which I want to place at the unix box machine inside cde directory.. at the following location that is /opt/app/cde/ now the credentials of unix box are abc345 -->(dummyid) ftyiu88--->(dummy passwd) ... (4 Replies)
Discussion started by: punpun66
4 Replies

2. Shell Programming and Scripting

Specify an entire UNIX command as a command line argument

I'm trying to write a bash script called YN that looks like the following YN "Specify a question" "doThis" "doThat" where "doThis" will be executed if the answer is "y", otherwise "doThat". For example YN "Do you want to list the file dog?" "ls -al dog" "" Here's my attempt... (3 Replies)
Discussion started by: LeoKSimon
3 Replies

3. UNIX Desktop Questions & Answers

Can Unix access Windows' File through Command Prompt in Unix

Hi all, I wish to know whether Unix can access window's file in Unix's terminal? Apart from that, how to copy files or share files between Window and Unix? I get to know of secure copy, however, my company's Unix does not support the feature of secure copy? Any other method for me to share/... (5 Replies)
Discussion started by: jessy83
5 Replies

4. UNIX for Dummies Questions & Answers

New to Unix command line and have a question about the "sort" command

I am going through the Unix Made Easy second edition book by John Muster. So far it's been very informative and I can tell it may be a bit out of date. In one of the exercises it talks about the "sort" command and using it to sort column's of data etc. The "sort" command has changed a bit and... (1 Reply)
Discussion started by: budfoxcat
1 Replies

5. UNIX for Advanced & Expert Users

unix command : how to insert text at the cursor location via command line?

Hi, Well my title isn't very clear I think. So to understand my goal: I have a script "test1" #!/bin/bash xvkbd -text blabla with xbindkeys, I bind F5 key in order it runs my test1 script So when I press F5, test1 runs. I'm under Emacs/Vi and I press F5 in order to have "blabla" be... (0 Replies)
Discussion started by: xib.be
0 Replies

6. Shell Programming and Scripting

assign a command line argument and a unix command to awk variables

Hi , I have a piece of code ...wherein I need to assign the following ... 1) A command line argument to a variable e.g origCount=ARGV 2) A unix command to a variable e.g result=`wc -l testFile.txt` in my awk shell script When I do this : print "origCount" origCount --> I get the... (0 Replies)
Discussion started by: sweta_doshi
0 Replies

7. HP-UX

How to use more than one MPE command STREAM with Unix command in a single shell?

Hello, I have problem in writing the shell script involving MPE command STREAM related to HP-UX and Unix command. Script is sh "nlshCMD 'STREAM <job name1>' | 'SHOWJOB' | grep $HPJOBNUM" sh "nlshCMD 'STREAM <job name2>' | 'SHOWJOB' | grep $HPJOBNUM" sh "nlshCMD 'STREAM <job name3>' |... (1 Reply)
Discussion started by: bosskr
1 Replies

8. Shell Programming and Scripting

How to use more than one MPE command STREAM with Unix command in a single shell?

Hello, I have problem in writing the shell script involving MPE command STREAM related to HP-UX and Unix command. Script is sh "nlshCMD 'STREAM <job name1>' | 'SHOWJOB' | grep $HPJOBNUM" sh "nlshCMD 'STREAM <job name2>' | 'SHOWJOB' | grep $HPJOBNUM" sh "nlshCMD 'STREAM <job name3>' |... (0 Replies)
Discussion started by: bosskr
0 Replies

9. UNIX for Dummies Questions & Answers

wc command help.... and other unix command....... urgent

hi all. thanks for looking i am doing some homework. one question is that when type wc and then how to tell the program that we have finished entering data? also why do some operating systems report 22 as the number of bytes in the file above, while others only 20? thanks so much,... (1 Reply)
Discussion started by: dashi2k
1 Replies
Login or Register to Ask a Question