I am new to shell scripting, i need your help to tackle a problem.
I have a single file, sample is below:
Code:
2008:07:08 07:01:14.360 (tid 4) INFO no bonus notifications to send
2008:07:08 07:01:50.823 (tid 1) INFO Database cleaned of all stale bonus records order than 30 days
2008:07:08 07:01:50.823 (tid 1) INFO No files matching the pattern were found
2008:07:08 07:02:14.770 (tid 4) INFO no bonus records to credit
2008:07:08 07:02:14.770 (tid 4) INFO no bonus notifications to send
2008:07:08 07:02:51.309 (tid 1) INFO Database cleaned of all stale bonus records order than 30 days
2008:07:08 07:02:51.309 (tid 1) INFO Found file: /var/ftp/emm/AnnivBonus_from_20080607_to_20080707_at_20080708_060001.txt
2008:07:08 07:02:51.316 (tid 1) INFO line [ msisdn = 133762, bonus = 56284, total usage = 562842 ]
2008:07:08 07:02:51.323 (tid 1) INFO Recorded bonus
2008:07:08 07:02:51.323 (tid 1) INFO line [ msisdn = 295361, bonus = 12315, total usage = 123153 ]
2008:07:08 07:02:51.324 (tid 1) INFO Recorded bonus
2008:07:08 07:02:51.324 (tid 1) INFO line [ msisdn = 130174, bonus = 16147, total usage = 161471 ]
2008:07:08 07:02:51.324 (tid 1) INFO Recorded bonus
2008:07:08 07:02:51.324 (tid 1) INFO line [ msisdn = 804610, bonus = 16287, total usage = 162873 ]
2008:07:08 07:02:51.324 (tid 1) INFO Recorded bonus
2008:07:08 07:02:51.324 (tid 1) INFO line [ msisdn = 781860, bonus = 26363, total usage = 263636 ]
2008:07:08 07:02:51.324 (tid 1) INFO Recorded bonus
2008:07:08 07:02:51.325 (tid 1) INFO line [ msisdn = 459960, bonus = 12342, total usage = 123426 ]
2008:07:08 07:02:51.325 (tid 1) INFO Recorded bonus
2008:07:08 07:02:51.325 (tid 1) INFO line [ msisdn = 910676, bonus = 17475, total usage = 174758 ]
Now the same file contains the following records
Code:
2008:07:08 07:03:20.655 (tid 4) INFO AIR credit operation success [ msisdn = 773210]
2008:07:08 07:03:20.656 (tid 4) INFO Bonus required [ msisdn = 113220, bonus = 18960, total usage = 189606 ]
2008:07:08 07:03:20.687 (tid 4) INFO AIR credit operation success [ msisdn = 113220]
2008:07:08 07:03:20.688 (tid 4) INFO Bonus required [ msisdn = 334050, bonus = 14184, total usage = 141849 ]
2008:07:08 07:03:20.719 (tid 4) INFO AIR credit operation success [ msisdn = 334050]
2008:07:08 07:03:20.720 (tid 4) INFO Bonus required [ msisdn = 893241, bonus = 64106, total usage = 641063 ]
2008:07:08 07:03:20.750 (tid 4) INFO AIR credit operation success [ msisdn = 893241]
2008:07:08 07:03:20.751 (tid 4) INFO Bonus required [ msisdn = 922615, bonus = 15175, total usage = 151756 ]
2008:07:08 07:03:20.781 (tid 4) INFO AIR credit operation success [ msisdn = 922615]
2008:07:08 07:03:20.782 (tid 4) INFO Bonus required [ msisdn = 150242, bonus = 14307, total usage = 143074 ]
2008:07:08 07:03:20.812 (tid 4) INFO AIR credit operation success [ msisdn = 150242]
2008:07:08 07:03:20.812 (tid 4) INFO Bonus required [ msisdn = 326001, bonus = 10709, total usage = 107092 ]
2008:07:08 07:03:20.842 (tid 4) INFO AIR credit operation success [ msisdn = 326001]
2008:07:08 07:03:20.842 (tid 4) INFO Bonus required [ msisdn = 132403, bonus = 18708, total usage = 187086 ]
2008:07:08 07:03:20.872 (tid 4) INFO AIR credit operation success [ msisdn = 132403]
2008:07:08 07:03:20.872 (tid 4) INFO Bonus required [ msisdn = 917950, bonus = 19989, total usage = 199896 ]
Now I want to get all the msisdns values (after = sign) from two different statements
INFO line [ msisdn = 130174, bonus = 16147, total usage = 161471 ]
INFO AIR credit operation success [ msisdn = 130174]
compare them, in case they are equal i want to output them like displayed below
Code:
20080708, 130174, 16147
.
.
.
The values that are not found in both sets of data can be output in an error file
Guys i need help to create this logix in UNIX/Solaris machine. So far i am unable to use the scripting knowledge that i have to perform this function.
So the essence of the problem is that when a USE for the credit (the "success" message) for a particular msisdn, you want to record all the "line" information for that msisdn. But the USE record comes AFTER the recording record. Is that the core of it?
I have a few questions...
1) Is this all being recorded moment by moment in one file like a log, or are the "USE" records in a separate file?
2) It makes sense that if there is a USE record where there has been no line record it would be an error.
3) Can you get two "line records" with the same msisdn without a USE record in between? If yes, you must account for this. How about the reverse? Two or mor uses for each recording? Either that is an error too, or you add up the credits or something, etc.
4) What happens if a line record never gets used? That is, how long do you hold (store) un-used "line records" before they become errors (if they ever do)?
I would start by storing the "line records" in a hash (perl). Basically you read along this file and every time you encounter a "line record" you store it using the msisdn as the index of the hash. As you come across a USE record (I'm assuming one log file here that is growing was you are reading it), you look for it's msisdn already in the hash (exists() in perl). If found, you have the line data as the value of the hash, and you can DELETE that value from the hash since it is accounted for, etc. That would be one way to approach it. If you don't have hashes, you will have to use an array, search it, etc. So long as the number of un-used line records stays low, performance shouldn't degrade.
$> ./check_msisdn a
Going to scan the file : a
msisdn mismatch found
cur_msisdni : 113220
prev_msisdn : 13220
and the contents of file a were the same that you have provided except for a change in the line
------------------------------------------------------------
2008:07:08 07:03:20.656 (tid 4) INFO Bonus required [ msisdn = 13220, bonus = 18960, total usage = 189606 ]
2008:07:08 07:03:20.687 (tid 4) INFO AIR credit operation success [ msisdn = 113220]
---------------------------------------------------------------
1) Is this all being recorded moment by moment in one file like a log, or are the "USE" records in a separate file?
Code:
Basically yes this is recorded moment by moment. But most of the time the file contains the polling information that is not required.
Point is the file contains this data as well as other garbage stuff which is not required.
Therefore you can consider the file as a static file which contains all the valid records and garbage values/lines.
2) It makes sense that if there is a USE record where there has been no line record it would be an error.
Code:
I can grep the data that i require into a seperate file meaning, the two lines
INFO line [ msisdn = 133762, bonus = 56284, total usage = 562842 ]
and
2008:07:08 07:03:20.655 (tid 4) INFO AIR credit operation success [ msisdn = 773210]
and do the processing.
3) Can you get two "line records" with the same msisdn without a USE record in between? If yes, you must account for this. How about the reverse? Two or mor uses for each recording? Either that is an error too, or you add up the credits or something, etc.
Code:
You see there is a chance that the two records INFO line and INFO AIR credit operation success may not match.
Because what i want to achieve is that the "recorded bonus (INFO line)" and the "credit operation success" are equal.
if any thing is missing just put it into an error file.
4) What happens if a line record never gets used? That is, how long do you hold (store) un-used "line records" before they become errors (if they ever do)?
Code:
This is why i am creating this script to see that both operations that is recording of bonuses and posting them are consistant.
Any descripencies could be seen in the error file.
Hi all,
We have a requirement like we need to create a program which will change a particular string in the file. For example
+=KA1238767 1121 3344645 686943 22356
01 567893 12435 12121 983627 121 1092 091217
02 may be for engine failure
In the above file we need to change the bold string... (2 Replies)
Hi,
Can anyone please help me on building the logic for writing a shell script which can delete blank lines from a file and count the number of duplicate lines in a file.
Thanks,
Indra (2 Replies)