Sponsored Content
Top Forums Shell Programming and Scripting Grep string from logs of last 1 hour on files of 2 different servers and calculate count Post 302450573 by poweroflinux on Friday 3rd of September 2010 06:11:42 AM
Old 09-03-2010
Question Grep string from logs of last 1 hour on files of 2 different servers and calculate count

Hi,
I am trying to grep a particular string from the files of 2 different servers without copying and calculate the total count of its occurence on both files.
File structure is same on both servers and for reference as follows:
Code:
27-Aug-2010 10:04:30,601|919122874903|phtunes_app|1282243292627|NotifySmsReception|DMGenerateLogInterceptor - ExternalTransactionID:SDP-DM-26713018, TransactionStatus:Requested
27-Aug-2010 10:05:30,601|919122874903|phtunes_app|1282243292627|NotifySmsReception|MaskingUnMaskingInterceptor - msisdn before masking 9122874903
27-Aug-2010 16:33:30,627|919122874903|phtunes_app|1282243292627|NotifySmsReception|MaskingUnMaskingInterceptor - msisdn after masking BJ#13340708
27-Aug-2010 16:34:30,637|BJ#13340708|phtunes_app|1282243292627|NotifySmsReception|CP URL:http://172.30.24.52/unitech_sms/unitechsms.php, AppInstanceIdhtunes_app, Keyword:busnews, SID:tel:BJ%2313340708, TransactionID:SDP-DM-26713018



---------- Post updated at 03:11 PM ---------- Previous update was at 03:09 PM ----------

Please consider this example for grep string from logs of last 15 minutes too.

---------- Post updated at 03:41 PM ---------- Previous update was at 03:11 PM ----------

Find the code below which i have made but getting error.
The code checks the last 15 minutes logs and search the string and get the total count.Can anybody correct the following code or provide me the optimized way:
Code:
bin/bash

to=`date +"%d-%b-%Y %T"`
echo $to
let from_in_seconds=`date +%s`-900
from=`date -d @$from_in_seconds +"%d-%b-%Y %T"`
echo $from

shortcodes=( "56882" "58585" "58888" "57575" "57677" );
for shortcode in ${shortcodes[@]}
do
    count=0
    sh_count=`awk '$0>=from && $0<=to' from="$from" to="$to" /opt/bea/ocsg_4.1/logs/customlogs/App_OP.log | grep "ShortCode=tel:${shortcode}" | wc -l`
    count=`expr $count + $sh_count`
    sh_count2=`ssh -n smehan@10.0.0.1 "awk '$0>=from && $0<=to' from=\"$from\" to=\"$to\" /opt/bea/ocsg_4.1/logs/customlogs/App_OP.log | grep \"ShortCode=tel:${shortcode}\" | wc -l"`
    count=`expr $count + $sh_count2`
    echo "${shortcode} : "$count
done

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to calculate busy hour

Dear All, please tell me any script which can able to calculate the bus hours by claculating the volume for example hour volume 1 100 2 200 3 300 4 7000 5 50 6 80 7 77 8 77 9 165 10 888 11 99 12 89 13 33 14 676 15 878 16 90 17 56 (14 Replies)
Discussion started by: shary
14 Replies

2. Shell Programming and Scripting

Count files every hour

Hi, I have a directory which uploads files every minute.I want to check the number of files uploaded for every one hour. I want to run a cron every hour to chk this but iam hanged like how to write the script. Like the script should count for one hr ie from 00:00 to 01:00 hrs then next hr... (3 Replies)
Discussion started by: nessj
3 Replies

3. Shell Programming and Scripting

Parse file from remote server to calculate count of string existence in that file

Hi I need to parse the file of same name which exist on different servers and calculate the count of string existed in both files. Say a file abc.log exist on 2 servers. I want to search for string "test" on both files and calculate the total count of search string's existence. For... (6 Replies)
Discussion started by: poweroflinux
6 Replies

4. UNIX for Dummies Questions & Answers

Rotate logs every 1 hour

Hello All, I am learning unix and basically I want to rotate one of my application logs every 1 hour. I need to rotate that file every one hour. I looked in the forums and googled.. but couldn;t get proper information. Requesting you all to kindly guide me. Our application is running on... (4 Replies)
Discussion started by: arunpvp
4 Replies

5. Shell Programming and Scripting

script to grep latest outofmemory string from the logs

I have requirement to prepare script which will grep for latest outofmemory message from the logs. I have used following command to grep the string from the logs,this script is not effective when logs are not getting updated as it will grep for old message. f=catalina.out var=`tail -10 $f |... (17 Replies)
Discussion started by: coolguyamy
17 Replies

6. Shell Programming and Scripting

Calculate total of log by hour

Hi, Just wondering, is there anyway I can get the total of logs generated by hours ? Let say I have these logs, Sep 23 04:48:43 hsbcufs: NOTICE: realloccg /: file system full Sep 23 04:48:47 hsbcufs: NOTICE: alloc: /: file system full Sep 23 04:48:51 hsbcufs: NOTICE: realloccg /: file... (14 Replies)
Discussion started by: dehetoxic
14 Replies

7. Shell Programming and Scripting

Finding/Grep on files with date and hour in the file name

Hi, I have a folder structure as follows, DATA -> 2012-01-01 -> 00 -> ABC_2012-01-03_00.txt -> 01 -> ABC_2012-01-03_01.txt -> 02 -> ABC_2012-01-03_02.txt ... -> 23 -> ABC_2012-01-03_02.txt -> 2012-01-02 -> 2012-01-03 So the dir DATA contains the above hierarchy, User input Start and... (6 Replies)
Discussion started by: mihirvora16
6 Replies

8. UNIX for Advanced & Expert Users

Finding/Grep on files with date and hour in the file name

Hi, I have a folder structure as follows, DATA -> 2012-01-01 -> 00 -> ABC_2012-01-03_00.txt -> 01 -> ABC_2012-01-03_01.txt -> 02 -> ABC_2012-01-03_02.txt ... -> 23 -> ABC_2012-01-03_02.txt -> 2012-01-02 ... (1 Reply)
Discussion started by: mihirvora16
1 Replies

9. Shell Programming and Scripting

Search string or words in logs without using Grep

I'm in need of some kind of script that will search for a string in each logfile in a directory but we don't want to use GREP. GREP seems to use up to much of our memory causing the server to use up a lot of swap space. Our log files are bigger than 500M on a daily basis. We lately started... (8 Replies)
Discussion started by: senormarquez
8 Replies

10. Shell Programming and Scripting

Grep a string and count following lines starting with another string

I have a large dataset with following structure; C 0001 Carbon D SAR001 methane D SAR002 ethane D SAR003 propane D SAR004 butane D SAR005 pentane C 0002 Hydrogen C 0003 Nitrogen C 0004 Oxygen D SAR011 ozone D SAR012 super oxide C 0005 Sulphur D SAR013... (3 Replies)
Discussion started by: Syeda Sumayya
3 Replies
DATETIME.__CONSTRUCT(3) 						 1						   DATETIME.__CONSTRUCT(3)

DateTime::__construct - Returns new DateTime object

       Object oriented style

SYNOPSIS
public DateTime::__construct NULL ([string $time = "now"], [DateTimeZone $timezone]) DESCRIPTION
Procedural style DateTime date_create NULL ([string $time = "now"], [DateTimeZone $timezone]) Returns new DateTime object. PARAMETERS
o $time -A date/time string. Valid formats are explained in Date and Time Formats. Enter NULL here to obtain the current time when using the $timezone parameter. o $timezone - A DateTimeZone object representing the timezone of $time. If $timezone is omitted, the current timezone will be used. Note The $timezone parameter and the current timezone are ignored when the $time parameter either is a UNIX timestamp (e.g. @946684800) or specifies a timezone (e.g. 2010-01-28T15:00:00+02:00). RETURN VALUES
Returns a new DateTime instance. Procedural style returns FALSE on failure. ERRORS
/EXCEPTIONS Emits Exception in case of an error. CHANGELOG
+--------+---------------------------------------------------+ |Version | | | | | | | Description | | | | +--------+---------------------------------------------------+ | 5.3.0 | | | | | | | If $time contains an invalid date/time format, | | | then an exception is now thrown. Previously an | | | error was emitted. | | | | +--------+---------------------------------------------------+ EXAMPLES
Example #1 DateTime.__construct(3) example Object oriented style <?php try { $date = new DateTime('2000-01-01'); } catch (Exception $e) { echo $e->getMessage(); exit(1); } echo $date->format('Y-m-d'); ?> Procedural style <?php $date = date_create('2000-01-01'); if (!$date) { $e = date_get_last_errors(); foreach ($e['errors'] as $error) { echo "$error "; } exit(1); } echo date_format($date, 'Y-m-d'); ?> The above examples will output: 2000-01-01 Example #2 Intricacies of DateTime.__construct(3) <?php // Specified date/time in your computer's time zone. $date = new DateTime('2000-01-01'); echo $date->format('Y-m-d H:i:sP') . " "; // Specified date/time in the specified time zone. $date = new DateTime('2000-01-01', new DateTimeZone('Pacific/Nauru')); echo $date->format('Y-m-d H:i:sP') . " "; // Current date/time in your computer's time zone. $date = new DateTime(); echo $date->format('Y-m-d H:i:sP') . " "; // Current date/time in the specified time zone. $date = new DateTime(null, new DateTimeZone('Pacific/Nauru')); echo $date->format('Y-m-d H:i:sP') . " "; // Using a UNIX timestamp. Notice the result is in the UTC time zone. $date = new DateTime('@946684800'); echo $date->format('Y-m-d H:i:sP') . " "; // Non-existent values roll over. $date = new DateTime('2000-02-30'); echo $date->format('Y-m-d H:i:sP') . " "; ?> The above example will output something similar to: 2000-01-01 00:00:00-05:00 2000-01-01 00:00:00+12:00 2010-04-24 10:24:16-04:00 2010-04-25 02:24:16+12:00 2000-01-01 00:00:00+00:00 2000-03-01 00:00:00-05:00 SEE ALSO
DateTime.createFromFormat(3), DateTimeZone.__construct(3), Date and Time Formats, date.timezone ini setting, date_default_time- zone_set(3), DateTime.getLastErrors(3), checkdate(3). PHP Documentation Group DATETIME.__CONSTRUCT(3)
All times are GMT -4. The time now is 04:51 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy