Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

script for get lines with specific time.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting script for get lines with specific time.
# 1  
Old 01-02-2012
Question script for get lines with specific time.
i want to make a script for grep any lines with key word and every time (5 min)

Ex. Log in Server.
Code:
.
.
.
03-01-2012 03:07:54,924 [1542133][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755292 | Message=Success
03-01-2012 03:09:13,789 [1542133][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755297 | Message=Success
03-01-2012 03:09:24,248 [1542133][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755298 | Message=Success
03-01-2012 03:09:29,561 [4994179][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755299 | Message=Success
03-01-2012 03:09:39,571 [4994179][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755300 | Message=Success
03-01-2012 03:09:51,855 [4994179][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755301 | Message=Success
03-01-2012 03:10:02,584 [1542133][][] - INFO MessageUtil - Return | Status=-109 | TxID=-1 | Message=Invalid Msn 
03-01-2012 03:10:19,647 [1542133][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755302 | Message=Success
03-01-2012 03:19:20,174 [4994124][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755374 | Message=Success
03-01-2012 03:19:48,806 [4994124][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755375 | Message=Success
03-01-2012 03:19:51,131 [4994124][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755376 | Message=Success
03-01-2012 03:19:59,532 [1542133][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755377 | Message=Success
03-01-2012 03:20:01,352 [1542133][][] - INFO MessageUtil - Return | Status=-109 | TxID=-1 | Message=Invalid Msn 
03-01-2012 03:20:08,015 [1542133][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755379 | Message=Success
03-01-2012 03:20:08,357 [1542133][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755381 | Message=Success
03-01-2012 03:21:01,969 [4994124][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755386 | Message=Success
03-01-2012 03:26:52,060 [1561474][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755414 | Message=Success
03-01-2012 03:27:01,605 [4994124][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755416 | Message=Success
03-01-2012 03:27:10,186 [1561474][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755417 | Message=Success
03-01-2012 03:28:14,890 [1561474][][] - INFO MessageUtil - Return | Status=0 | TxID=12010300000548755419 | Message=Success
.
.
.

i want to output.
Ex.
Keyword
Code:
ATTEMP = grep  "Status"
SUCCESS grep  "Status=0"
Fail = grep -v "Status=0"

Output.
Code:
DATE            TIME            ATTEMP        SUCCESS        FAIL
03-01-2012     03:05:00              0                0        0
03-01-2012     03:10:00              6                6        0
03-01-2012     03:15:00              2                1        1
03-01-2012     03:20:00              4                4        0
03-01-2012     03:25:00              4                3        1
03-01-2012     03:30:00              4                4        0

please help me!
Last edited by ooilinlove; 01-03-2012 at 08:56 AM..
# 2  
Old 01-02-2012
First, you need show us what you have tried.
# 3  
Old 01-03-2012
if you have not yet started on this project or looking at a brick wall, here is a quick breakdown of what needs to happen ...

A. grab entries from the log in 5 minute increments for each hour for each day

B. sort entries according to categories

C. create and format report


i suspect that you are getting stuck with item A so here's something to help you out ... i did it using ksh in solaris 10 so you may need to modify it according to the shell you need to use and the OS you are working in ... you still need to include the parts for items B and C but that should be easier to do ... the script may run faster using perl, python or ruby so feel free to use and modify ... be sure to test in safe place before running in production area ...

btw, "`< $tmpfile`" construct does not work in some shells so use "`cat $tmpfile`" instead ...

anyways, good luck!

Code:
#! /bin/ksh 

list=/tmp/999
tmpfile=/tmp/444".tmp"
tmpdate=/tmp/444".date"
tmphour=/tmp/444".hour"
tmpatt=/tmp/444".att"
tmpok=/tmp/444".ok"
tmpno=/tmp/444".no"

grabentry(){
start=$1
end=$2

while [ $start -le $end ]
do
    eval "awk '\$1 ~ /$date/ && \$2 ~ /$hour:$start:/ {print \$0}'" $list
    start=`expr $start + 1`
    if [ $start -lt 10 ]
    then
        start=0$start
    fi
done
}


rm /tmp/444.* 2> /dev/null

## 
## uncomment next 7 lines if script performance slows
#awk -F" " '{print $1 | "sort -u"}' $list > $tmpdate
#awk -F" " '{print $2}' $list | awk -F":" '{print $1 | "sort -u"}' > $tmphour
#
#for date in `< $tmpdate`
#do
#    for hour in `< $tmphour`
#    do
for date in `awk -F" " '{print $1 | "sort -u"}' $list`                                #<--| comment out if above 7 lines are uncommented
do                                                                                    #<--|                                                                          
    for hour in `awk -F" " '{print $2}' $list | awk -F":" '{print $1 | "sort -u"}'`   #<--|
   do                                                                                 #<--|                                                                                                 
        end=4
        while [ $end -le 59 ]
        do
            if [ $end -lt 10 -a $end -ne 00 ]
            then
                end=0$end
            fi
            start=`expr $end - 4`
            if [ $start -eq 0 ]
            then
                start=00
            fi
            if [ $start -lt 10 -a $start -ne 00 ]
            then
                start=0$start
            fi
            grabentry $start $end | awk 'NF > 0' > $tmpfile.$date.$hour.$end 
            if [ ! -s $tmpfile.$date.$hour.$end ]
            then
                rm $tmpfile.$date.$hour.$end 2> /dev/null
            else
                echo "-- cutoff time is $date $hour.$end --"  ## echo line used for debugging, can be removed
                cat $tmpfile.$date.$hour.$end                 ## replace this line with code to process, create and format report
            fi
            end=`expr $end + 5`
        done
    done
done

exit 0

# 4  
Old 01-03-2012
I tried it because I am still a novice unix.
I think it is.

Code:
time.log
.
.
00:0[0-9]
00:1[0-9]
00:2[0-9]
00:3[0-9]
00:4[0-9]
00:5[0-9]
.
.

my script
Code:
DATE=`date "+%Y%m%d"`
LOGPATH="/home/nagios/"
LOGNAME="Status.log"
LOG=$LOGPATH$LOGNAME
LOGMON="/home/nagios/logmon.log"
echo "$LOG"
date="02-01-2012 "

dates=DATE
times=TIME
attemp=ATTEMP
success=SUCCESS
fail=FAIL

for li in `cat time.log`
do

a1=`cat Status.log  |grep "$date$li" |wc -l`
a2=`cat Status.log  |grep "$date$li" |grep "Status=0" |wc -l`
a3=`cat Status.log  |grep "$date$li" |grep -v "Status=0" |wc -l`

echo "$date     $times  $a1     $a2     $a3" >> logmon.log
done

while true
        do
                clear
                echo " Monitor  Success "
                echo "Server name = `uname -n`"
                echo " -------------------------------------------------------------------------------------------------------------------------"
                echo "$dates    $times  $attemp $success        $fail" |awk '{printf "%15s %15s %15s %15s %15s\n", $1,$2,$3,$4,$5}'
                tail -30 $LOGMON |awk '{printf "%15s %15s %15s %15s %15s\n", $1,$2,$3,$4,$5}'
                sleep 100 
        done

my script output.
Code:
 Monitor  Success 
Server name = Abidamaru
 -------------------------------------------------------------------------------------------------------------------------
           DATE            TIME          ATTEMP         SUCCESS            FAIL
     02-01-2012            TIME              48              46               2
     02-01-2012            TIME              91              88               3
     02-01-2012            TIME              55              53               2
     02-01-2012            TIME              25              25               0
     02-01-2012            TIME              38              36               2
     02-01-2012            TIME              67              66               1
     02-01-2012            TIME              48              46               2

I have a problem, it displays real time.
And display the date and time. I wanted it to come out in this format.

Code:
DATE                                 TIME   
2012-01-04           00:50:00
2012-01-04           01:00:00
2012-01-04           01:10:00

# 5  
Old 01-03-2012
1st, your desired output is in 10 minute increments ...

2nd, just replace the actual time with the cutoff time increment (i.e., 00:50:00 replaces 00:49:54,245) when formatting the report ... you should be able to use awk with this matter ...

3rd, use awk to reorder the date
# 6  
Old 01-03-2012
Quote:
Originally Posted by Just Ice
1st, your desired output is in 10 minute increments ...

2nd, just replace the actual time with the cutoff time increment (i.e., 00:50:00 replaces 00:49:54,245) when formatting the report ... you should be able to use awk with this matter ...

3rd, use awk to reorder the date
Please show an example.
I began to study awk.
# 7  
Old 01-03-2012
please see samples and discussions from the search results ...
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Quick UNIX command to display specific lines in the middle of a file from/to specific word

This could be a really dummy question. I have a log text file. What unix command to extract line from specific string to another specific string. Is it something similar to?: more +/"string" file_name Thanks (4 Replies)
Discussion started by: aku
4 Replies

2. Shell Programming and Scripting

Removing specific lines from script files.

Hello, Activity to perform: 1. Find all of the "*.tmp" files in a given user directory 2. Determine which ones have "find" in them. 3. Replace the "find sequence" of commands with a "list set" of commands. Example: Original file: -------------- define lastn1 = "A" define... (7 Replies)
Discussion started by: manishdivs
7 Replies

3. Shell Programming and Scripting

Script to run commands at a specific time.

Hello All, I have written a script which which is working fine to a certain logic of it. But i want a part of the script to run two commands at 00:10 hrs every day. These two command are 1. rm -rf /path/to/folder 2. mail the content of a file. How do i achieve this. Thanks. ... (4 Replies)
Discussion started by: Siddheshk
4 Replies

4. Shell Programming and Scripting

script for get lines with specific date

I'm a newbie in AIX, i want to make a script for grep any lines with date bellow 20 PRINT0089-88615 data1 3072 Mon Dec 19 17:53:49 WITA 2011 PRINT0089-88616 data1 4096 Mon Dec 19 17:53:49 WITA 2011 PRINT0089-88618 data1 5120 Mon Dec 19... (7 Replies)
Discussion started by: michlix
7 Replies

5. UNIX for Dummies Questions & Answers

Script required to truncate all the lines except a specific snippet.

Hi, I have a file with the following structure. XXXXX........... YYYYY........... ................. .................. ZZZZZZ...... qwerty_start.............. .................. ................. .................. querty_end................ .............................. (3 Replies)
Discussion started by: abinash
3 Replies

6. Shell Programming and Scripting

Ending script at specific time.

Hello Everybody.. I've written the script that kick off through CRON job and kill itself by specific time. I've start time and end time specify in env file. i.e START_TIME=1500 (03:00 PM) END_TIME=0600 (06:00 AM) It always works good if my START_TIME is before midnight and my... (4 Replies)
Discussion started by: nirav_soni
4 Replies

7. UNIX for Dummies Questions & Answers

command for selecting specific lines from a script

I need help on following script: I need to print the lines which are in bold letters in separate file as record string("|") emp_name; string("|") emp_id; decimal("|") emp_salary; string("|") emp_status; string("\n") emp_proj; end (1 Reply)
Discussion started by: gardasgangadhar
1 Replies

8. UNIX for Advanced & Expert Users

How to delete specific lines at the same time

Dear All I have a pattern which look like this: 2 20080312_10:55:35.800 Spain-Telefonica ISC 9 IAM 927535957 34670505334 f 275 COT b 700 ACM b 6577 CPG b 10726 ANM b 202195 REL f 202307 RLC :COMMA: NCI=15,FCI=2101,CPC=0A,TMR=00,USI,OFI=00: :COMMB: BCI=0214,OBI=01,ACT: :RELCAUSE:10: This... (1 Reply)
Discussion started by: zanetti321
1 Replies

9. Shell Programming and Scripting

need help--script to filter specific lines from multiple txt files

Hi folks, - I have 800 txt files - those files are cisco router configs router1.txt router2.txt ... router800.txt I want to accomplish the following: - I want to have a seperate file with all the filenames that I want to process - I want a script that goes trough all those... (7 Replies)
Discussion started by: I-1
7 Replies

10. Shell Programming and Scripting

Shell Script to read specific lines in a file

I have a file with contents as follows Record 1: Rejected - Error on table "DWO"."P2G_CUST_EVENTS". ORA-00001: unique constraint (DWO.CUST_EVENTS_PK) violated Record 5: Rejected - Error on table "DWO"."P2G_CUST_EVENTS". ORA-00001: unique constraint (DWO.CUST_EVENTS_PK) violated Record 6:... (5 Replies)
Discussion started by: varshanswamy
5 Replies
Login or Register to Ask a Question