Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Strip timestamps from logs using sed?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Strip timestamps from logs using sed?
# 1  
Old 03-02-2009
Question Strip timestamps from logs using sed?
Hullo. I have a log file:

Code:
(I) 02/03/2009 12:31:01 [12345] : Service Started
(I) 02/03/2009 12:31:02 [12345] : XML:
<xml>
   <stuff>
   ...
</xml>
(I) 02/03/2009 12:31:02 [12345] : Service Stoped
...

...and I would like to remove date/time/PID-stamps so I can create a baseline for future comparisson. Ideal output:

Code:
(I) [DATE] [TIME] [PID] : Service Started
(I) [DATE] [TIME] [PID] : XML:
<xml>
   <stuff>
   ...
</xml>
(I) [DATE] [TIME] [PID] : Service Stoped

I guess psuedocode would be as follows:

Code:
For each line that starts with a "(", replace characters 5-23 with "[DATE] [TIME]", and replace "['one or more numbers']" with "[PID]". Print all other lines as is.

Any ideas?

Thanks in advance Smilie
# 2  
Old 03-02-2009
awk:
Code:
awk '{
            if(index( $0, "(" )==1)  {
                       printf("%s [DATE] [TIME] [PID] ", $1) 
                       
                       for(i=5; i<=NF; i++ ) {printf("%s ", $i) } print ""
            }
            else { print $0}
       }'  logfile > newfile

Last edited by jim mcnamara; 03-02-2009 at 06:48 AM..
# 3  
Old 03-02-2009
EDIT: Erroneous ")" - seems to work with it removed!

Thanks Jim - figured awk might be my best bet, but your code is giving me some grief and I'm not sure why...

Code:
awk: cmd. line:4:                        for(i=5; i<=NF; i++ ) {printf"%s ", $i) } print ""
awk: cmd. line:4:                                                              ^ syntax error

# 4  
Old 03-02-2009
Great pointers - the first character in brackets can actually vary i.e. (I)/(W)/(D), but I've dusted off my sed/awk reference and will let you know how I get on! Smilie
# 5  
Old 03-02-2009
I changed the code slightly to fix the error - and handled the first field problem as well.

see above
# 6  
Old 03-02-2009
Code:
use strict;
open FH,"<a.txt";
while(<FH>){
	if(/^\(/){
		my @arr=split(/ +/,$_);
		@arr[1..3]=('[DATE]','[TIME]','[PID]');
		print join " ",@arr;
	}
	else{
		print $_;
	}
}

Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

If I ran perl script again,old logs should move with today date and new logs should generate.

Appreciate help for the below issue. Im using below code.....I dont want to attach the logs when I ran the perl twice...I just want to take backup with today date and generate new logs...What I need to do for the below scirpt.............. 1)if logs exist it should move the logs with extention... (1 Reply)
Discussion started by: Sanjeev G
1 Replies

2. Shell Programming and Scripting

Fetching timestamps from the logs.

Dear Experts, I need some help to get the time stamps from the Unix logs. Basically I am want to analyze which step is taking more time. Please give me some inputs. Thanks a lot for your help. I have got different codes for each for phase as we can see in the logs 00002 – UPDT Starting... (5 Replies)
Discussion started by: srikanth38
5 Replies

3. Shell Programming and Scripting

To print lines between 2 timestamps using awk|sed and regex

Hi, I am using the following code to fetch lines that are generated in last 1 hr . Hence, I am using date function to calculate -last 1 hr & the current hr and then somehow use awk (or sed-if someone could guide me better) with some regex pattern. dt_1=`date +%h" "%d", "%Y\ %l -d "1 hour... (10 Replies)
Discussion started by: sarah-alikhan31
10 Replies

4. Shell Programming and Scripting

sed script to parse logs issue

I have this script to parse some logs: #!/bin/bash id=$1 shift sed "/(id=$id)/,/^$/!d" "$@" Usage: ./script.sh 1234 logfile The logs have an empty line before the logged events/timestamps -- most of the time. And this is my issue, since when there is no empty line, it will catch things... (4 Replies)
Discussion started by: KidCactus
4 Replies

5. UNIX for Dummies Questions & Answers

Compare 2 timestamps

Hi, i have current timestamp, lets say "12:02:45" in an variable (var1) and another timestamp "08:30:00" fetched from table in another variable2 (var2). How do i compare 2 timestamps in unix shell scripting. if var 1 > var 2 then echo message. Thanks in advance. (3 Replies)
Discussion started by: prasannarajesh
3 Replies

6. Shell Programming and Scripting

strip the square brackets from a word using sed or var expansion

Hi, I am trying to strip out leading and trailing brackets from a word. for example, I have a value, in a file. I want to strip out the leading and trailing brackets it and should get the value Running. I am using the following statement but in vain. grep "Workflow run status:" <... (4 Replies)
Discussion started by: svajhala
4 Replies

7. Shell Programming and Scripting

Grep yesterday logs from weblogic logs

Hi, I am trying to write a script which would go search and get the info from the logs based on yesterday timestamp and write yesterday logs in new file. The log file format is as follows: """"""""""""""""""""""""""... (3 Replies)
Discussion started by: harish.parker
3 Replies

8. Shell Programming and Scripting

timestamps

Hello! I have the following problem. I read a file using perl, each line of this file has the fllowing format. 14/4/2008 8:42:03 πμ|10800|306973223399|4917622951117|1||1259|1|126|492|433||19774859454$ Th first field is the timestamp and the second field is the offset in seconds. How can... (1 Reply)
Discussion started by: chriss_58
1 Replies

9. Shell Programming and Scripting

unsing sed to strip html tags - help

Hi, I am trying to strip html tags of a string for example <TD>no problem</TD> the sesult should be no problem but could never get rid off all the tags sed 's/<..D>//g' Please help, I am new (3 Replies)
Discussion started by: zap
3 Replies
Login or Register to Ask a Question