Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Help building the logic for a script

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help building the logic for a script
# 1  
Old 07-08-2008
Help building the logic for a script
Hi guys,

I am new to shell scripting, i need your help to tackle a problem.
I have a single file, sample is below:

Code:
2008:07:08 07:01:14.360 (tid 4) INFO no bonus notifications to send
2008:07:08 07:01:50.823 (tid 1) INFO Database cleaned of all stale bonus records order than 30 days
2008:07:08 07:01:50.823 (tid 1) INFO No files matching the pattern were found
2008:07:08 07:02:14.770 (tid 4) INFO no bonus records to credit
2008:07:08 07:02:14.770 (tid 4) INFO no bonus notifications to send
2008:07:08 07:02:51.309 (tid 1) INFO Database cleaned of all stale bonus records order than 30 days
2008:07:08 07:02:51.309 (tid 1) INFO Found file: /var/ftp/emm/AnnivBonus_from_20080607_to_20080707_at_20080708_060001.txt
2008:07:08 07:02:51.316 (tid 1) INFO line [ msisdn = 133762, bonus = 56284, total usage = 562842 ]
2008:07:08 07:02:51.323 (tid 1) INFO Recorded bonus
2008:07:08 07:02:51.323 (tid 1) INFO line [ msisdn = 295361, bonus = 12315, total usage = 123153 ]
2008:07:08 07:02:51.324 (tid 1) INFO Recorded bonus
2008:07:08 07:02:51.324 (tid 1) INFO line [ msisdn = 130174, bonus = 16147, total usage = 161471 ]
2008:07:08 07:02:51.324 (tid 1) INFO Recorded bonus
2008:07:08 07:02:51.324 (tid 1) INFO line [ msisdn = 804610, bonus = 16287, total usage = 162873 ]
2008:07:08 07:02:51.324 (tid 1) INFO Recorded bonus
2008:07:08 07:02:51.324 (tid 1) INFO line [ msisdn = 781860, bonus = 26363, total usage = 263636 ]
2008:07:08 07:02:51.324 (tid 1) INFO Recorded bonus
2008:07:08 07:02:51.325 (tid 1) INFO line [ msisdn = 459960, bonus = 12342, total usage = 123426 ]
2008:07:08 07:02:51.325 (tid 1) INFO Recorded bonus
2008:07:08 07:02:51.325 (tid 1) INFO line [ msisdn = 910676, bonus = 17475, total usage = 174758 ]

Now the same file contains the following records

Code:
2008:07:08 07:03:20.655 (tid 4) INFO AIR credit operation success [ msisdn = 773210]
2008:07:08 07:03:20.656 (tid 4) INFO Bonus required [ msisdn = 113220, bonus = 18960, total usage = 189606 ]
2008:07:08 07:03:20.687 (tid 4) INFO AIR credit operation success [ msisdn = 113220]
2008:07:08 07:03:20.688 (tid 4) INFO Bonus required [ msisdn = 334050, bonus = 14184, total usage = 141849 ]
2008:07:08 07:03:20.719 (tid 4) INFO AIR credit operation success [ msisdn = 334050]
2008:07:08 07:03:20.720 (tid 4) INFO Bonus required [ msisdn = 893241, bonus = 64106, total usage = 641063 ]
2008:07:08 07:03:20.750 (tid 4) INFO AIR credit operation success [ msisdn = 893241]
2008:07:08 07:03:20.751 (tid 4) INFO Bonus required [ msisdn = 922615, bonus = 15175, total usage = 151756 ]
2008:07:08 07:03:20.781 (tid 4) INFO AIR credit operation success [ msisdn = 922615]
2008:07:08 07:03:20.782 (tid 4) INFO Bonus required [ msisdn = 150242, bonus = 14307, total usage = 143074 ]
2008:07:08 07:03:20.812 (tid 4) INFO AIR credit operation success [ msisdn = 150242]
2008:07:08 07:03:20.812 (tid 4) INFO Bonus required [ msisdn = 326001, bonus = 10709, total usage = 107092 ]
2008:07:08 07:03:20.842 (tid 4) INFO AIR credit operation success [ msisdn = 326001]
2008:07:08 07:03:20.842 (tid 4) INFO Bonus required [ msisdn = 132403, bonus = 18708, total usage = 187086 ]
2008:07:08 07:03:20.872 (tid 4) INFO AIR credit operation success [ msisdn = 132403]
2008:07:08 07:03:20.872 (tid 4) INFO Bonus required [ msisdn = 917950, bonus = 19989, total usage = 199896 ]

Now I want to get all the msisdns values (after = sign) from two different statements
INFO line [ msisdn = 130174, bonus = 16147, total usage = 161471 ]
INFO AIR credit operation success [ msisdn = 130174]

compare them, in case they are equal i want to output them like displayed below

Code:
20080708, 130174, 16147
.
.
.

The values that are not found in both sets of data can be output in an error file

Guys i need help to create this logix in UNIX/Solaris machine. So far i am unable to use the scripting knowledge that i have to perform this function.



Regards,
# 2  
Old 07-08-2008
So the essence of the problem is that when a USE for the credit (the "success" message) for a particular msisdn, you want to record all the "line" information for that msisdn. But the USE record comes AFTER the recording record. Is that the core of it?

I have a few questions...

1) Is this all being recorded moment by moment in one file like a log, or are the "USE" records in a separate file?

2) It makes sense that if there is a USE record where there has been no line record it would be an error.

3) Can you get two "line records" with the same msisdn without a USE record in between? If yes, you must account for this. How about the reverse? Two or mor uses for each recording? Either that is an error too, or you add up the credits or something, etc.

4) What happens if a line record never gets used? That is, how long do you hold (store) un-used "line records" before they become errors (if they ever do)?

I would start by storing the "line records" in a hash (perl). Basically you read along this file and every time you encounter a "line record" you store it using the msisdn as the index of the hash. As you come across a USE record (I'm assuming one log file here that is growing was you are reading it), you look for it's msisdn already in the hash (exists() in perl). If found, you have the line data as the value of the hash, and you can DELETE that value from the hash since it is accounted for, etc. That would be one way to approach it. If you don't have hashes, you will have to use an array, search it, etc. So long as the number of un-used line records stays low, performance shouldn't degrade.
# 3  
Old 07-09-2008
need few clarifications
Hi,
I need few clarifications in the requirements.
In the file, are we supposed to check only the lines that contains

"INFO Bonus required "

to the next line that contains

"INFO AIR credit operation success "???

If yes, we can go for a simpe script like


-----------------------------------------------------------------------

if [ $# -lt 1 ]
then
echo "Usage : " $0 "<file_name>"
exit 1
fi

echo "Going to scan the file : "$1

file_name=`echo $1`

prev_line=2

for cur_line in `cat $file_name | tr ' ' '+'`
do

echo $prev_line | grep "INFO+Bonus+required+" >test_file

if [[ -s test_file ]]
then
prev_msisdn=`echo $prev_line | tr ',' '+' | tr -s '+' | cut -d '+' -f 11`
cur_msisdn=`echo $cur_line | tr ']' '+' | tr -s '+' | cut -d '+' -f 13`

if [ "$prev_msisdn" -ne "$cur_msisdn" ]
then
echo "msisdn mismatch found"
echo "cur_msisdni : "$cur_msisdn
echo "prev_msisdn : "$prev_msisdn
fi
fi


prev_line=`echo $cur_line`

done
-----------------------------------------------------------------------




which works like

$> ./check_msisdn a
Going to scan the file : a
msisdn mismatch found
cur_msisdni : 113220
prev_msisdn : 13220


and the contents of file a were the same that you have provided except for a change in the line
------------------------------------------------------------
2008:07:08 07:03:20.656 (tid 4) INFO Bonus required [ msisdn = 13220, bonus = 18960, total usage = 189606 ]
2008:07:08 07:03:20.687 (tid 4) INFO AIR credit operation success [ msisdn = 113220]
---------------------------------------------------------------
# 4  
Old 07-09-2008
@ quine

1) Is this all being recorded moment by moment in one file like a log, or are the "USE" records in a separate file?
Code:
Basically yes this is recorded moment by moment. But most of the time the file contains the polling information that is not required. 
Point is the file contains this data as well as other garbage stuff which is not required. 
Therefore you can consider the file as a static file which contains all the valid records and garbage values/lines.

2) It makes sense that if there is a USE record where there has been no line record it would be an error.
Code:
I can grep the data that i require into a seperate file meaning, the two lines
INFO line [ msisdn = 133762, bonus = 56284, total usage = 562842 ]

and

2008:07:08 07:03:20.655 (tid 4) INFO AIR credit operation success [ msisdn = 773210]

and do the processing.


3) Can you get two "line records" with the same msisdn without a USE record in between? If yes, you must account for this. How about the reverse? Two or mor uses for each recording? Either that is an error too, or you add up the credits or something, etc.
Code:
You see there is a chance that the two records INFO line and INFO AIR credit operation success may not match.
Because what i want to achieve is that the "recorded bonus (INFO line)" and the "credit operation success" are equal. 
if any thing is missing just put it into an error file.


4) What happens if a line record never gets used? That is, how long do you hold (store) un-used "line records" before they become errors (if they ever do)?
Code:
This is why i am creating this script to see that both operations that is recording of bonuses and posting them are consistant. 
Any descripencies could be seen in the error file.



@snowline84

The lines we need to check are:

INFO line
and
INFO AIR credit operation success
Login or Register to Ask a Question

Previous Thread | Next Thread

2 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Need help in building Unix script

Hi all, We have a requirement like we need to create a program which will change a particular string in the file. For example +=KA1238767 1121 3344645 686943 22356 01 567893 12435 12121 983627 121 1092 091217 02 may be for engine failure In the above file we need to change the bold string... (2 Replies)
Discussion started by: susant.igate
2 Replies

2. Shell Programming and Scripting

Help required on building the logic

Hi, Can anyone please help me on building the logic for writing a shell script which can delete blank lines from a file and count the number of duplicate lines in a file. Thanks, Indra (2 Replies)
Discussion started by: igandu
2 Replies
Login or Register to Ask a Question