Issue faced while accessing data from files on different servers.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Issue faced while accessing data from files on different servers.
# 1  
Old 09-08-2010
Question Issue faced while accessing data from files on different servers.

I have two log files of same name and structure which resides on different servers with ssh-keygen installed.
I want to search for a list of strings available in an array on last 15 minutes logs of both files and calculate the total count of occurence of each string.

Sample format of both file is same and as follows:
Code:
08-Sep-2010 07:58:40,044|8923629811|aclwireless_app|1282498070897|NotifySmsReception|CP URL to be called : http://203.122.58.164/uninorintegrat...413#5853#102#0#
08-Sep-2010 07:58:41,672|9126657443|aclwireless_app|1282498070897|NotifySmsReception|CP URL to be called : http://203.122.58.164/uninorintegrat...761#1445#102#0#
08-Sep-2010 09:47:52,778|9122796686|internal_app|1283918818906926|SendSms|ChargeAmount|ChargingInterceptor - subscriber details processed sucessfully- {arg0.referenceCode=channelType:SMS;transactionId:1283918818906926;pricePtAvl:true;PPL_FLAG:FALSE;CorrelatorId:BEA1-611DC04C995F;contentType:Text;senderMSISDN:52211;contentId:1096;Subscriber_ID:BJ#12101396;SPACCOUNT_ID:PhoneyTunes;methodName:sendSms;Sub_Profile:Pre-Paid;IMSI:405876120820301;Customer_Retrieved:true;contentName:PT_JOKES;cpID:Phoneytunes;origContentOwnerId:10225;serviceId:PT_JOKES;Circle_Name:BJ;CustomerID:B_12318157;Circle_ID:4;isRated:No;MSISDN:9122796686;productName:JKGEN001;basePrice:-1.0, arg0.endUserIdentifier=9122796686, arg0.charge.description=CP is Calling TO This API, arg0.charge.currency=INR, arg0.charge.code=code0, arg0.charge.amount=0}
08-Sep-2010 09:47:52,789|7398965380|subscription_app|-7932740d%3A12aedfb6d15%3A-26d3|ChargeAmount|GetBalance|PaymentPlugin-Response -  Retrieved Balance Bucket: 1;20091003;20110306;122.50;|
08-Sep-2010 09:47:52,790|7398965380|subscription_app|-7932740d%3A12aedfb6d15%3A-26d3|ChargeAmount|Exception: [Message:null]
08-Sep-2010 09:47:52,810|9122796686|internal_app|1283918818906926|SendSms|ChargeAmount|DMGenerateLogInterceptor - InternalTransactionID:SDP-DM-31297182, ExternalTransactionID:1283918818906926, TransactionStatus:Sent For Delivery, INStatus:FAILED
08-Sep-2010 09:47:52,813|9122796686|internal_app|1283918818906926|SendSms|ChargeAmount|CustomCDRInterceptor - CDR Info[Optional_Field1:30.0,Subscription_Channel:,Optional_Field2:Subscribe,Transaction_ID:SDP-DM-31297182,Content_ID:1096,IMEI:,Product_Name:JKGEN001,PPL_FLAG:FALSE,Charge_Code:code0,Base_Price:0.0,CustomerID:B_12318157,Circle_Name:BJ,Sender_MSISDN:52211,IMSI:405876120820301,Content_Status:,Location:BJ,Circle_ID:4,Original_Content_Owner_ID:10225,CPNAME:PhoneyTunes,Content_Price:0,Zone:,Content_Name:PT_JOKES,Static_ID:BJ#12101396,External_Correlation_Id:1283918818906926,Subscription_Type:,MSISDN:9122796686,Transaction_Mode:SMS,Transaction_DateTime:2010-09-08 09:47:52 GMT+05:30,Content_Type:Text,Sub_Profile:Pre-Paid,CPID:Phoneytunes,Other_Info:]
08-Sep-2010 09:47:52,820|9122796686|internal_app|1283918818906926|SendSms|ChargeAmount|PaymentPlugin-Request -  [amountToBeCharged-0 | balanceChargeDetails0;0;0;0;| | transactionId-null | profileType-Pre-Paid ]
08-Sep-2010 09:47:52,823|9122796686|internal_app|1283918818906926|SendSms|ChargeAmount|DMGenerateLogInterceptor - InternalTransactionID:SDP-DM-31297182, ExternalTransactionID:1283918818906926, TransactionStatus:Sent For Delivery, INStatus:Success
08-Sep-2010 09:47:52,823|9122796686|internal_app|1283918818906926|SendSms|ChargeAmount|Response Parameters:[Result:Success]
08-Sep-2010 09:47:52,970|BJ#8748783|phtunes_app||SendSms|Request Params:[Address:tel:BJ#8748783,Message:Joke:  Jack: What are you going to be when you have finished studying and passing all your exams. Buck: Probably an old age pensioner.                          ,SenderName:52211,ChargingInformation:[Code:code0,Currency:INR,Description:CP is Calling TO This API,Amount:0],SimpleReference:[],X-Params:[[Key:transactionId,Value:1283918819184906],[Key:origContentOwnerId,Value:10225],[Key:channelType,Value:SMS],[Key:cpID,Value:Phoneytunes],[Key:serviceId,Value:PT_JOKES],[Key:PPL_FLAG,Value:FALSE],[Key:contentType,Value:Text],[Key:contentName,Value:PT_JOKES],[Key:senderMSISDN,Value:52211],[Key:isRated,Value:No],[Key:contentId,Value:1096]]]
08-Sep-2010 09:47:52,970|BJ#8748783|phtunes_app||SendSms|ValidateMSISDNInterceptor - sid to be validated BJ#8748783

In the above logs i want to search a list of strings which are available in an array as follows:
Code:
shortcodes=( "56882" "58585" "58888" "57575" "57677" );

Now I want to search the string contains "ShortCode=tel:"$shortcode from the last 15 minutes logs and calculate their count accordingly.For example
Code:
56882 : 12
58585 : 15
58888 : 18
57575 : 12
57677 : 98

The above output includes sum of count matching the string from both files on different servers.
The code what i have written is as follows and giving error while accessing file on other server:
Code:
#!/bin/bash
to=`date +"%d-%b-%Y %T"`
echo $to
let from_in_seconds=`date +%s`-90000
from=`date -d @$from_in_seconds +"%d-%b-%Y %T"`
echo $from
shortcodes=( "52211" );
for shortcode in ${shortcodes[@]}
do
    count=0
    sh_count=`awk '$0>=from && $0<=to && $0~"ShortCode=tel:" s' from="$from" to="$to" s="$shortcode" App_OP.log | wc -l`
    count=`expr $count + $sh_count`
    sh_count2=`ssh -n root@10.0.0.1 "awk '$0>=from && $0<=to && $0~"ShortCode=tel:" s' from="$from" to="$to" s="$shortcode" App_OP.log | wc -l"`
    count=`expr $count + $sh_count2`
    echo "${shortcode} : "$count
done

Please guide me for the above and if possible provide me more optimized way of doing above.
Thanks in advance

Last edited by poweroflinux; 09-08-2010 at 08:18 AM.. Reason: Incorrect printing
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Solaris

Solaris OS having issue accessing LUN from Qlogic swich

Hi Community, I am facing one issue related storage accessing from solaris 10 machine(M5000). it was working fine then we shifted to another site. the only change on that site is FC switch. The issue i am facing is from solaris machine i can able to see the LUN, it;s only a test lun of 30... (17 Replies)
Discussion started by: bentech4u
17 Replies

2. Shell Programming and Scripting

Issue with files when there is no data in a file.

I have two files File1.txt 000199458 000199463 000200442 000200831 000200866 000201009 000201050 000201405 000201666 000201682 File2.txt (3 Replies)
Discussion started by: pingiliarjun
3 Replies

3. Shell Programming and Scripting

Issue with accessing value inside while loop, outside it

Hi, GetName() { if then echo " Please enter the name: " read Name tempvar=0 while read line do if then tempvar=`expr $tempvar + 1` echo $tempvar ... (10 Replies)
Discussion started by: rituparna_gupta
10 Replies

4. Web Development

Accessing a Perl CGI script, security issue

Hi Everybody, I was wondering if it was possible for someone to gain access to my Perl CGI scripts before they are interpreted by Perl (mod_perl on apache2) i.e. getting a hold of my raw scripts and not the html output? Let's say I use the DBI module where I have the hostname, user and... (2 Replies)
Discussion started by: z1dane
2 Replies

5. SuSE

Regarding accessing multiple servers using single public ip address

Hello, Currently we are having different linux servers (for example: let's assume audio server, video server and text server) to handle requests from outside users. Suppose the outside users in different LAN (Local Area Network), other than the servers. For example user is in 20 series LAN and... (5 Replies)
Discussion started by: navneet_2009
5 Replies

6. Programming

Structure element accessing issue...

Hi all, Can someone advice, why I'm getting a segmentation fault on a Linux system for a program which looks like the below one--: typedef struct idev{ int p, char q; } Idev; typedef struct abc{ int i; Idev *idev_ptr; } ABC; int main(){ ABC a_var; char str; int... (4 Replies)
Discussion started by: Praveen_218
4 Replies

7. UNIX for Advanced & Expert Users

concurrency issue while Accessing Mail Box from shell script

Hi Bros, I am in a serious trouble with concurrency issue while using mailx. I have a shell script which reads all the emails of a unix user account and create a copy (another mbox for processing and archive purpose). This script works fine for 99.99% of the time but sometime it start creating... (2 Replies)
Discussion started by: Sumit_Fundoo
2 Replies

8. Shell Programming and Scripting

Script for accessing data..

Well i m kinda newbie at this shell scripting stuff .. so spare me if u find me total stupid As a matter of fact i need a script to access a remote unix server via telnet , for time being assume any ip address , username and password , then on that server script should check the path ( assume... (1 Reply)
Discussion started by: Dastard
1 Replies

9. UNIX for Dummies Questions & Answers

Accessing UNIX Data over LINUX

Hello! Unfortunatly i'm totaly new with UNIX and LINUX... Actually Problem is i have some data on floppy disk which was written on UNIX system 5 version now i have installed LINIX ver 9 is there any way i can access unix data on linux 9? if so please reply me with commands i shall be very... (2 Replies)
Discussion started by: xohaib
2 Replies
Login or Register to Ask a Question