Script (ksh) to get data in every 30 mins interval for the given date
Hello,
Since I m new to shell, I had a hard time to sought out this problem.
I have a log file of a utility which tells that batch files are successful with timestamp. Given below is a part of the log file.
Kindly help me with a ksh to get data from this log file in 30 minutes interval i.e for the given date in every 30 mins interval I need how many batches
were successful.
Desired output:
Last edited by Don Cragun; 10-04-2013 at 04:51 AM..
Reason: Fix ending CODE tag
Since I m new to shell, I had a hard time to sought out this problem.
I have a log file of a utility which tells that batch files are successful with timestamp. Given below is a part of the log file.
Kindly help me with a ksh to get data from this log file in 30 minutes interval i.e for the given date in every 30 mins interval I need how many batches
were successful.
Desired output:
[CODE] Run the script in background like nohup ksh scriptname & it will run ever 30 minutes and send you email.
I agree with pravin27 that your input data has out of range timestamps. It is also weird that you want the final entry in your output to have timestamps 23:29:59 and 23:59:59 rather than 23:30:00 and 23:59:59. But, following you general pattern and ignoring the out of range data, the following script seems to do what you want:
producing the output:
when the user types in the data in red in response to the prompt for the date and the file named log contains your sample input data.
This User Gave Thanks to Don Cragun For This Post:
I agree with pravin27 that your input data has out of range timestamps. It is also weird that you want the final entry in your output to have timestamps 23:29:59 and 23:59:59 rather than 23:30:00 and 23:59:59. But, following you general pattern and ignoring the out of range data, the following script seems to do what you want:
producing the output:
when the user types in the data in red in response to the prompt for the date and the file named log contains your sample input data.
Thanks a lot for your solution.... Regrets for the typo...
can you please explain the code such that it will be helpful for me to build on this... for example if I want to match some different patterns or strings.
1st line: Use the Korn shell to interpret this script
2nd line: Print a prompt asking the user to input a date.
3rd line: Read the date the user enters and save it in the shell variable named dd.
4th line: Invoke the awk utility telling it to use space and colon characters as field delimiters and define the awk variable dd to have the same value as the shell variable dd.
5th, 6th, and 7th lines: Before any input is read by the awk script, define the awk variable fmt to be the format string we will use to print results in the end.
8th line: If the 1st field on each input file line is the same string as the dd awk variable and the last field on the line is the string "successful", then increment the element of the array s[] indexed by two subscripts (the 2nd field in the line [the hour] with the leading zero removed if there is one, and 0 if the 3rd field in the line [the minute] is less than or equal to 29 or 1 if the minute is 30 or larger).
9th, 10th, and 11th line: After all input lines have been read, loop through every half hour with h set to the hour of the day and m set to the half hour within the hour and print the user entered date (dd), the hour as a two digit string with leading 0 fill, the starting minute as a two digit string with leading 0 fill, the hour again, the ending minute, and the numberer of times the patterns matched on the 8th line were found on lines in the input file within that half hour period.
12 line: Terminate the awk script and specify that the input to be processed is contained in a file named log.
Last edited by Don Cragun; 10-04-2013 at 09:57 AM..
Reason: Fix line number typo
This User Gave Thanks to Don Cragun For This Post:
1st line: Use the Korn shell to interpret this script
2nd line: Print a prompt asking the user to input a date.
3rd line: Read the date the user enters and save it in the shell variable named dd.
4th line: Invoke the awk utility telling it to use space and colon characters as field delimiters and define the awk variable dd to have the same value as the shell variable dd.
5th, 6th, and 7th lines: Before any input is read by the awk script, define the awk variable fmt to be the format string we will use to print results in the end.
8th line: If the 1st field on each input file line is the same string as the dd awk variable and the last field on the line is the string "successful", then increment the element of the array s[] indexed by two subscripts (the 2nd field in the line [the hour] with the leading zero removed if there is one, and 0 if the 3rd field in the line [the minute] is less than or equal to 29 or 1 if the minute is 30 or larger).
9th, 10th, and 11th line: After all input lines have been read, loop through every half hour with h set to the hour of the day and m set to the half hour within the hour and print the user entered date (dd), the hour as a two digit string with leading 0 fill, the starting minute as a two digit string with leading 0 fill, the hour again, the ending minute, and the numberer of times the patterns matched on the 8th line were found on lines in the input file within that half hour period.
12 line: Terminate the awk script and specify that the input to be processed is contained in a file named log.
Great thanks for your explanation.
I was trying out this piece of code below
Because i wanted make that script very generic such that in which ever place the pattern "successful" is found it should get the count. Since I want to use the same script for two more diffrent log in which the successful message can be in any place not neccesarily in the last ($NF == "successful").
But the problem i was facing here was i was not able to increment the timestamp values i.e "st" and "et" with 30 minutes and get a logic for that to repeat the piece of code for every 30 minutes.
Hi All,
I have an issue which I'm trying to understand a way of doing, I have several nodes which contain syslog events which I want to force trigger an email initially (eventually leading to another method of alerting but to start with an email).
Basically the syslog file will have hours worth... (6 Replies)
Hi
I searched this forum before posting the question, but couldnt find it, the issue
i'm facing is, i'm trying to select a column from a netezza table from a korn
shell script, but the query runs
var=$(nzodbcsql -q "select MAX(millcount) from table1";)
echo $var
it returns the value like... (10 Replies)
Hi...
I am very new to shell scripting. I have written a script with help of this forum and some googling and it works the way I want it to. Currently this script checks for my SIP trunk registration every 5 seconds, if registration is not available then it reboots my router through telnet... (4 Replies)
A happy Monday to you all,
I have a .csv file which contains data taken every 5 seconds. I want to average these 5 second data points into 30 minute averages!
date co2
25/06/2011 08:04 8.31
25/06/2011 08:04 8.32
25/06/2011 08:04 8.33... (18 Replies)
I am trying to write a script that will allow me to train others with commands that I run manually by only allowing the exact command before continuing onto the next set of commands. Here is where I come into an issue. I have changed the directories for this post.
Software we run creates files... (2 Replies)
Hi i am writing a cron job.
so for it i need the 60 days old date form current date in variable.
Like today date is 27 jan 2011 then output value will be stote in variable in formet Nov 27.
i am using EST date, and tried lot of solution and see lot of post but it did not helpful for me. so... (3 Replies)
Hello,
I need help about a shell script
I have a text file with this fields:
2009/01/19 09:33:35: --> ---ORA-28817: PL/SQL function returned an error.
2009/01/19 09:33:35: --> ---ORA-28817: PL/SQL function returned an error.
2009/01/19 09:33:35: --> ---ORA-28817: PL/SQL function returned an... (4 Replies)
I want a shell script which extract data from a log file which contains date and time-wise data and i need the data for a perticular interval of time...what can i do??? (3 Replies)
Hi All,
I have a xml file for example as described below
<xml>
<address>
<street><street>
<address/>
<isbn>426728783932020308393930303</isbn>
<book>
<name>
</name>
</book>
.
.
.
</xml>
My problem is to get the isbn number from the above described file using ksh script. Could... (6 Replies)