Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Parsing and timestamp a pattern in log
Top Forums Shell Programming and Scripting Parsing and timestamp a pattern in log Post 302785731 by amazigh42 on Tuesday 26th of March 2013 09:39:24 AM
Old 03-26-2013
Parsing and timestamp a pattern in log
Hello

Thanks to Chubler_XL and MadeInGermany for their help few weeks ago.
Now, i would like modifying the script, see the next POST.

The old script works like that :
I picked any random hours
In the logs there is the stamp time of webservices, i can see the behavior or errors of webservices. Just put the approximate times in order to have a portion of the log. This could also handle errors when the script does not find the right hour. The script greps a bit of script between timestamp.

Code:
cat log_name

Code:
aaaaaaaaaaaaaa
bbbbbbbbbbbbb
cccccccccccc
[24/01/2013 09:10]
sssssssssssssss
error-jonas123
nnnnnnnnnnnnn
[24/01/2013 10:10]
uuuuuuuuuuuuuuu
jjjjjjjjjjjjjj
error-jonas123
mmmmmmmmmmmmm
[24/01/2013 10:30]
oooooooooooo
error-jonas123
qqqqqqqqqqq
[24/01/2013 10:45]
vvvvvvvvv
sssssssss
wwwwwwwwww

The result
Code:
./my script log_name
[24/01/2013 10:10]
uuuuuuuuuuuuuuu
jjjjjjjjjjjjjj
error-jonas123
mmmmmmmmmmmmm
[24/01/2013 10:30]

Of course, it would be desirable to put the dates in variables.
Code:
vi my_script

Code:
#!/bin/bash
log_name=$1
if [[ "$log_name" =~ .gz$ ]]
     then z_cat="gunzip -c"
     else z_cat=cat
fi
$z_cat $log_name |awk -F"[/ \\\][]" -v S="24/01/2013 10:10" -v E="24/01/2013 10:30" '
function dcmp(b) {
  if($4>b[3])return  1;
  if($4<b[3])return -1;
  if($3>b[2])return  1;
  if($3<b[2])return -1;
  if($2>b[1])return  1;
  if($2<b[1])return -1;
  if($5>b[4])return  1;
  if($5<b[4])return -1;
  return 0;
}
BEGIN{split(S, ds, "[/ ]"); split(E, de, "[/ ]") }
/^[[][0-9][0-9]\/[0-1][0-9]\/[[0-9][0-9][0-9][0-9] / {
   if(s&&dcmp(de)>=0) {print; exit}
   if(!s&&dcmp(ds)<=0) {f=x;w=1}
   if(!s&&dcmp(ds)>=0) {printf "%s",f; f=x; s=1 }
}
!w&&!s {f=f $0 "\n"}
s'

---------- Post updated at 08:39 AM ---------- Previous update was at 08:32 AM ----------

Now, i would like modifying the script like this :
The script has several identical patterns like for example error-jonas123.
The script will have to pick the first pattern, then it will have to search the nearest date before. Then it will have to search the last pattern and it will have to search the nearest date after.

Code:
cat log_name

Code:
aaaaaaaaaaaaaa
bbbbbbbbbbbbb
cccccccccccc
[24/01/2013 09:10]
sssssssssssssss
error-jonas123
nnnnnnnnnnnnn
[24/01/2013 10:10]
uuuuuuuuuuuuuuu
jjjjjjjjjjjjjj
error-jonas123
mmmmmmmmmmmmm
[24/01/2013 10:30]
oooooooooooo
error-jonas123
qqqqqqqqqqq
[24/01/2013 10:45]
vvvvvvvvv
sssssssss
wwwwwwwwww

Code:
my_script log_name

The expected result
Code:
[24/01/2013 09:10]
sssssssssssssss
error-jonas123
nnnnnnnnnnnnn
[24/01/2013 10:10]
uuuuuuuuuuuuuuu
jjjjjjjjjjjjjj
error-jonas123
mmmmmmmmmmmmm
[24/01/2013 10:30]
oooooooooooo
error-jonas123
qqqqqqqqqqq
[24/01/2013 10:45]

Can you give me somes ideas to change the script.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Spooling a log file with timestamp

Hi From shell script i am invoking sqlplus to connect to oracle database and then i spool a csv file as with output. What i want to do is to change the file name with timestamp on it so after spooling finish shell script change file name with time stamp. can someone help me to do that . Thanks... (2 Replies)
Discussion started by: ukadmin
2 Replies

2. Shell Programming and Scripting

List all log records logged after $timestamp ?

I am trying to find a way to list every records inside a file (usually a log file) that are present after a record mathing/greater-then a timestamp supplied by another script. The timestamp can be anywhere inside the record and it is usually in the standard `date` format (will not look for other... (5 Replies)
Discussion started by: Browser_ice
5 Replies

3. Shell Programming and Scripting

concatenate log file lines up to timestamp

Hi, Using sed awk or perl I am trying to do something similar to https://www.unix.com/shell-programming-scripting/105887-sed-awk-concatenate-lines-until-blank-line-2.html but my requirement is slightly different. What I am trying to accomplish is to reformat a logfile such that all lines... (4 Replies)
Discussion started by: AlanC
4 Replies

4. Shell Programming and Scripting

Delete log file entries based on the Date/Timestamp within log file

If a log file is in the following format 28-Jul-10 ::: Log message 28-Jul-10 ::: Log message 29-Jul-10 ::: Log message 30-Jul-10 ::: Log message 31-Jul-10 ::: Log message 31-Jul-10 ::: Log message 1-Aug-10 ::: Log message 1-Aug-10 ::: Log message 2-Aug-10 ::: Log message 2-Aug-10 :::... (3 Replies)
Discussion started by: vikram3.r
3 Replies

5. Shell Programming and Scripting

Parsing a timestamp until EOF

hey guys, i'm having a bit of trouble with my script. based on the time you started a shift for work, it's supposed to read the input, and then parse a log file of alerts accordingly. my issue is parsing out from the time entered as your start time until the end of the file. the format looks like... (6 Replies)
Discussion started by: terrell
6 Replies

6. Shell Programming and Scripting

pattern parsing

Can somebody show me an example of of using either '#' or '?' to remove part of string. I am reading files from directories and I want to check if file ends with *.log *.dmp or begins with a arch_* I DONT want to use AWK or SED to do this since there maybe 1000's of files I need to test... (4 Replies)
Discussion started by: BeefStu
4 Replies

7. UNIX for Dummies Questions & Answers

Need help finding a file where a pattern exists and the file has a timestamp

So, I know how to do some of this stuff on an individual level, but I'm drawing a blank as to how to put it all together. I have a pattern that I'm looking for in a log file. The log file I know came in yesterday, so I want to limit the search to that day's listing of files. How would I do... (5 Replies)
Discussion started by: kontrol
5 Replies

8. UNIX for Dummies Questions & Answers

How to compare a file by its timestamp and store in a different location whenever timestamp changes?

Hi All, I am new to unix programming. I am trying for a requirement and the requirement goes like this..... I have a test folder. Which tracks log files. After certain time, the log file is getting overwritten by another file (randomly as the time interval is not periodic). I need to preserve... (2 Replies)
Discussion started by: mailsara
2 Replies

9. Shell Programming and Scripting

To check timestamp in logfile and display lines upto 3 hours before current timestamp

Hi Friends, I have the following logfile. Currently time in india is 07/31/2014 12:33:34 and i have the following content in logfile. I want to display only those entries which contain string 'Exception' within last 3 hours. In this case, it would be the last line only I can get the... (12 Replies)
Discussion started by: srkmish
12 Replies

10. Shell Programming and Scripting

Help with parsing file with combination of pattern

I have a file1 like prt1|als28.1 prt3|als53.1 prt2|als550.1 prt1|bls9.2 prt2|als7.2 prt2|bls0.2 prt2|als872.1 prt1|bls871.1 prt2|als6.2 prt4|als22.1 prt2|bls43.2 I want to create a file2 from this file by comparing all the possible combinations of patterns (prt) assuming prt1... (3 Replies)
Discussion started by: sammy777
3 Replies
All times are GMT -4. The time now is 09:38 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy