Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

sed 2 hours in log

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sed 2 hours in log
# 8  
Old 02-12-2013
Quote:
Originally Posted by Chubler_XL
Thanks for your Solaris corrections MadeInGermany, nice to know that {n,n} regs don't work with nawk.

I'm not quite sure what the requirement is around this last like, the following change
will include the final date line that matches/exceeds the Ending date/time

Replace:
Code:
if(s&&dcmp(de)>=0) exit

With:
Code:
if(s&&dcmp(de)>=0) {print; exit}

That's great. Smilie
Thanks to Chubler_XL and MadeInGermany for this superb bit of script.
I'm a beginner, i can not understand some lines, can you help me decipher the blue lines. Smilie

Code:
awk -F"[/ ]" -v S="24/01/2013 10:10" -v E="24/01/2013 10:30" '
function dcmp(b) {
  if($3>b[3])return  1;
  if($3<b[3])return -1;
  if($2>b[2])return  1;
  if($2<b[2])return -1;
  if($1>b[1])return  1;
  if($1<b[1])return -1;
  if($4>b[4])return  1;
  if($4<b[4])return -1;
  return 0;
}
BEGIN{split(S, ds, "[/ ]"); split(E, de, "[/ ]") }
/^[0-9][0-9]\/[0-1][0-9]\/[0-9][0-9][0-9][0-9] / {
   if(s&&dcmp(de)>=0) {print; exit}
   if(!s&&dcmp(ds)<=0) {f=x;w=1}
   if(!s&&dcmp(ds)>=0) {printf "%s",f; f=x; s=1 }
}
!w&&!s {f=f $0 "\n"}
s' server.log

# 9  
Old 02-12-2013
Quote:
Originally Posted by amazigh42
That's great. Smilie
Thanks to Chubler_XL and MadeInGermany for this superb bit of script.
I'm a beginner, i can not understand some lines, can you help me decipher the blue lines. Smilie
if($3>b[3])return 1;
if($3,b[3])return -1;
...
 if 3rd field (year) of current line is greater than 3rd field in input parameter (year of start or end date) return 1 (ie > 0).
As you can see years (field 3) are compared first and only if the two year values are the same are month values (filed 2) compared, then days (filed 1) and finally time (field 4).
The final result is that dcmp() returns 0 if dates are identical,1 if current line is greater than input param, and -1 if current line is less than input param.

/^[0-9][0-9]\/[0-1][0-9]\/[0-9][0-9][0-9][0-9] /
Match lines that start with two digits followed by slash then 0 or 1 then digit then slash and finally 4 digits (ie line starts with a 99/99/9999 format date).

 if(s&&dcmp(de)>=0) {print; exit}
if start date already found (s is non blank/zero) and date in current line >= date-end then print current line and stop processing.

 if(!s&&dcmp(ds)<=0) {f=x;w=1}
if start date not found yet (s is not set) and date in current line <= date-start, blank contents of var f and set w flag. f is a buffer that contains all lines found from start of document.

 if(!s&&dcmp(ds)>=0) {printf "%s",f; f=x; s=1 }
if start date not found yet (s is not set) and date in current line >= date-start, print f buffer, blank it and set start date found flag (s=1).

!w&&!s {f=f $0 "\n"}
if w and s flags are not set append current line to the f buffer

s
if s flag is set print current line
Last edited by Chubler_XL; 02-12-2013 at 05:04 AM..
# 10  
Old 02-12-2013
What an limpid explaination ! It's clearer for me, thank you.

I have a particular case, when i have a gzip file like server.log.gz
I tried to use zcat in the script.
An error has occured.
Fatal can't open zcat server.log.gz
What's the problem ?

Code:
awk -F"[/ ]" -v S="24/01/2013 10:10" -v E="24/01/2013 10:30" '
function dcmp(b) {
  if($3>b[3])return  1;
  if($3<b[3])return -1;
  if($2>b[2])return  1;
  if($2<b[2])return -1;
  if($1>b[1])return  1;
  if($1<b[1])return -1;
  if($4>b[4])return  1;
  if($4<b[4])return -1;
  return 0;
}
BEGIN{split(S, ds, "[/ ]"); split(E, de, "[/ ]") }
/^[0-9][0-9]\/[0-1][0-9]\/[0-9][0-9][0-9][0-9] / {
   if(s&&dcmp(de)>=0) {print; exit}
   if(!s&&dcmp(ds)<=0) {f=x;w=1}
   if(!s&&dcmp(ds)>=0) {printf "%s",f; f=x; s=1 }
}
!w&&!s {f=f $0 "\n"}
s' "zcat server.log.gz"

Last edited by amazigh42; 02-12-2013 at 09:31 AM..
# 11  
Old 02-12-2013
If awk has no file parameter it will process stdin so pipe output of zcat to awk like this:

Code:
 zcat server.log.gz | awk -F"[/ ]" -v S="24/01/2013 10:10" -v E="24/01/2013 10:30" '...code here...'

# 12  
Old 02-12-2013
On traditional Unix zcat is linked to the traditional uncompress rather than gunzip.
In this case replace zcat with gunzip -c.
# 13  
Old 02-12-2013
Here no problems, just solutions.
Thanks the guys.
I would like to modify the script.
I want to use either cat or gunzip following if it is a single file or gzip file.
Have you ideas about conditions in red because it didn't work ?

Code:
file=$1
if [ -f *.gz] 
     then $command="gunzip -c" 
     else $command=cat
fi
$command $file | awk -F"[/ ]" -v S="24/01/2013 10:10" -v E="24/01/2013 10:30" '
function dcmp(b) {
  if($3>b[3])return  1;
  if($3<b[3])return -1;
  if($2>b[2])return  1;
  if($2<b[2])return -1;
  if($1>b[1])return  1;
  if($1<b[1])return -1;
  if($4>b[4])return  1;
  if($4<b[4])return -1;
  return 0;
}
BEGIN{split(S, ds, "[/ ]"); split(E, de, "[/ ]") }
/^[0-9][0-9]\/[0-1][0-9]\/[0-9][0-9][0-9][0-9] / {
   if(s&&dcmp(de)>=0) {print; exit}
   if(!s&&dcmp(ds)<=0) {f=x;w=1}
   if(!s&&dcmp(ds)>=0) {printf "%s",f; f=x; s=1 }
}
!w&&!s {f=f $0 "\n"}
s

Last edited by amazigh42; 02-12-2013 at 04:29 PM..
# 14  
Old 02-12-2013
Code:
if [ -f $file.gz ] 
     then command="gunzip -c" 
     else command=cat
fi

1. You do want to test for $file.gz, don't you.
2. The shell [ ] wants a space on both sides. If you like you can rephrase it
Code:
if test -f $file.gz

3. The shell uses var= in assignment, and $var in reference. (While perl uses $var in both assignment and reference.)
This User Gave Thanks to MadeInGermany For This Post:
amazigh42
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing log file for last 2 hours

I want to parse a log file which i am grepping root user connection but is showing whole day and previous day detail as well. First i want to see last 2 hours log file then after that i want to search particular string. Lets suppose right now its 5:00PM, So i want to see the log of 3:00PM to... (6 Replies)
Discussion started by: learnbash
6 Replies

2. Shell Programming and Scripting

Log search and mail it if the log is updated before 24 hours from the current time

Hi , We have around 22 logs , each has different entries. I have to automate this using shell script. The ideas which am sharing is given below 1) We use only TAIL -100 <location and name of the log> Command to check the logs. 2) We want to check whether the log was updated before 24... (13 Replies)
Discussion started by: Kalaihari
13 Replies

3. Shell Programming and Scripting

Need help looking for missing hours.

I have a file that should cover a days worth of stats, at the beginning of each 15 minute report I have a unique header that looks like the below example. The "0000" and "0015" will change in the header line to show which 15 Minute interval the report is covering and of course from day to day the... (7 Replies)
Discussion started by: fsanchez
7 Replies

4. Shell Programming and Scripting

ps -ef |grep 24 hours

I need to grep PIDs older than 24 hours (1 day) or more. ps -ef |grep ??? Please advise. (10 Replies)
Discussion started by: Daniel Gate
10 Replies

5. AIX

cron off by 5 hours

stupid question im sure, but its frustrating My cron jobs are off by 5 hours. My system time is right but all of my cron jobs are running approximately 5 hours late. Any idea why? (4 Replies)
Discussion started by: mshilling
4 Replies

6. Shell Programming and Scripting

CurrentTime-4 hours

Hi, Good Afternoon! I am writing this script on "sh" and have Variables as below. #Time in hours ex: 09 JobTime=`echo $StartTime |awk '{print $2}'|cut -f1 -d':'` SystemHours=`date +%H` How can go 4 hours back for each variable in a day? Another Question? JobStat=`dsjob -report... (5 Replies)
Discussion started by: rajubollas
5 Replies

7. What is on Your Mind?

How Many hours on Computer?

How many hours you spend on Computer in a day??? (10 Replies)
Discussion started by: malcomex999
10 Replies

8. Shell Programming and Scripting

how to list files between last 6 hours to 3 hours

Hi Frens, I want to list some files from a directory, which contains "DONE" in their name, i am receiving files every minute. In this i want to list all the files which are newer than 6 hours but older than 3 hours, of current time i dont want my list to contain the latest files which are ... (4 Replies)
Discussion started by: Prat007
4 Replies

9. Shell Programming and Scripting

Last 24 hours of a log file

I'm looking to pull the last 24 hours of a log file. Here's what I've got so far: yesterday=$(TZ=$TZ+24 date +"%b %e %H:%M") today=$(date +"%b %e %H:%M") echo $yesterday $today grep -E "^$yesterday|^$today" /var/adm/syslog/syslog.log But that pulls everything from $yesterday from... (1 Reply)
Discussion started by: Bert
1 Replies

10. Shell Programming and Scripting

move log files over 12 hours old...

Hi, I know I can use touch and find's "! -newer" option to list files that are older than a specific time, but what is a good way to get a list of files that are over 12 hours old? The log pruner will run throughout the day, twice an hour. So I can't easily use a cronjob touch command to generate... (1 Reply)
Discussion started by: Thomas Pluck
1 Replies
Login or Register to Ask a Question