Match the amount fields in the source file vs trigger file


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Match the amount fields in the source file vs trigger file
# 1  
Old 12-22-2011
Match the amount fields in the source file vs trigger file

Hello,

I have to write a script to compare the sum of the amount fields in a source file and the amount field in another file.

details are:
based on the name of the source file (say SALES as an example), a file already available in a path will be grabbed (say SALES_ParmFile) and this file will have a text - "amount fields" - 2,3 and 5. this means the source file SALES.txt has amount values in the fields 2 , 3 and 5. there is a trigger file (SALES_TRIGGER) sent with the source file everyday which will have the total of the amount fields, see below.. I need to match the source file amount fields and the trigger file total amounts and for no match write to a log file.

example:

cat SALES_ParmFile
amount fields: 2,3,5

cat SALES
Q,100,120,100,23,SEW
A,10,34,56,48,ABR

cat SALES_TRIGGER:
SALES_AMOUNT2:110
SALES_AMOUNT3:154
SALES_AMOUNT5:71


as you can notice, the field number in the SALES_ParmFile file will help to get the corresponding amount field.. the total of amount field 2 in the source file has to be compared with the SALES_AMOUNT2 field value and similarly total of amount field 5 to be compared with SALES_AMOUNT5.

the thing here is everything is dynamic.. there are 3 fields that has amount fields in a source SALES and in other file it can be 4 fields and the same script has to be used for all the files..

can somebody help me with this? if you have questions, please let me know.


thanks in advance
# 2  
Old 12-23-2011
Ugly way .. Anyway got the result .. Smilie
Code:
a=$(nawk -F: '/amount fields:/{print "cut -d, -f"$2" < SALES"}' SALES_ParmFile|sh|
    nawk -F, '{for(i=1;i<=NF;i++)A[i]+=$i}END{for(i in A)printf A[i]" ";print ""}'| tr ' ' '\n'|grep '.'|sort -n|xargs)
 
b=$(nawk -F: '/amount fields:/{gsub(/,/," ",$2);gsub(/ /,"|SALES_AMOUNT",$2);print "egrep \""$2"\" SALES_TRIGGER"}' SALES_ParmFile|sh|
      awk -F: '{print $2|"sort -n|xargs"}')
 
[ "$a" == "$b" ] && echo "Matches" || echo "Not Matches"

Result may vary if input file changes ..
# 3  
Old 12-27-2011
thanks Jayan Jay but unfortunately I dont have nawk here.. so I can't use the script.. can you please help me with this using any other command?

---------- Post updated at 06:44 PM ---------- Previous update was at 11:00 AM ----------
# 4  
Old 12-27-2011
Hi
I don't get your requirement .
Can you please elaborate by providing input data and require output ?

Thanks
# 5  
Old 12-28-2011
Hello,
I have to write a script to compare the sum of the amounts in a data file and the amount field value in an another file.
there is a table in which I have the list of source system name, source files and amount field names etc. Below is the data for a source system. "ABZ" has to be passed a parameter to the script.

PHP Code:
Query:
select Source_sys_name,Source_file_name,amount_field_names from source_sys_det where Source_sys_name='ABZ'
 
Result:
Source_sys_name,Source_file_name,amount_field_names
ABZ
,SALES,sales_amount|cost_price|profit|Loss 
(there are many source files for ABZ, but for an example I have mentioned only SALES here)

in the output above, in column 3, there are 4 fields - sales_amount|cost_price|profit|Loss which are amount field names. this means the source file ABZ_SALES has amount values in these 4 fields and we have to compare the amount values in a file with the source file ABZ_SALES available in a unix directory based on the field names in both the files.
Source file ABZ_SALES has 9 columns. the first line in the file is header and it is a pipe delimited file.
PHP Code:
cat ABZ_SALES:
sales_amount|cost_price|profit|Loss|state
10
|12|13|30|ca
12
|3|4|42|mn 
Now, we have the list of amount fields names and the values of them in the source file, ABZ_SALES.
Below is the trigger file which will be sent everyday along with the source file mentioned above.
Below is the content of the trigger file for ABZ:
PHP Code:
cat ABZ_Trigger_file
SALES
sales_amount in total
-22
cost_price in total
-15
profit in total
-17
loss in total
-72

INVENTORY
a in total
-200
b in total
-108
c in total
-1009
e in total
-2007
d in total
-120
f in total
-304 
In the file above, under SALES section, you can see the total for the individual amount fields. this amount should be compared with the amount in the ABZ_SALES file (for column sales_amount, as per ABZ_Trigger_file amount is 22 and in the ABZ_SALES file, sum the amount for the sales_amount field which is 10+12 and it is equal to 22 in the ABZ_Trigger_file file. If the amounts don't match, then it has to be written to a error_file and the comparison to proceed for other columns until all the column comparison are done until the loss column.
Similar to the source file SALES this comparison has to be done for INVENTORY also. and this has 2 amount fields, 3 weight fields and 1 unit field. like wise, a source system ABZ can contain multiple source files which is dynamic.
EMC source system has 5 files. and each of these files have amount fields, weight fields and unit fields in different locations and have to be compared and if the comparison fails, it has to be written to an error file else the script will complete successfully.
For INVENTORY:
PHP Code:
Query:
select Source_sys_name,Source_file_name,amount_field_namesfrom source_sys_det where Source_sys_name='ABZ' and 
Source_file_name='INVENTORY'
Result:
Source_sys_name,Source_file_name,amount_field_names
ABZ
,INVENTORY,a|
thanks
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Matching two fields in two csv files, create new file and append match

I am trying to parse two csv files and make a match in one column then print the entire file to a new file and append an additional column that gives description from the match to the new file. If a match is not made, I would like to add "NA" to the end of the file Command that Ive been using... (6 Replies)
Discussion started by: dis0wned
6 Replies

2. Shell Programming and Scripting

awk to update file based on match in 3 fields

Trying to use awk to store the value of $5 in file1 in array x. That array x is then used to search $4 of file1 to find aa match (I use x to skip the header in file1). Since $4 can have multiple strings in it seperated by a , (comma), I split them and iterate througn each split looking for a match.... (2 Replies)
Discussion started by: cmccabe
2 Replies

3. Shell Programming and Scripting

awk move select fields to match file prefix in two directories

In the awk below I am trying to use the file1 as a match to file2. In file2 the contents of $5,&6,and $7 (always tab-delimited) and are copied to the output under the header Quality metrics. The below executes but the output is empty. I have added comments to help and show my thinking. Thank you... (0 Replies)
Discussion started by: cmccabe
0 Replies

4. UNIX for Beginners Questions & Answers

Match Fields between two files, print portions of each file together when matched in ([g]awk)'

I've written an awk script to compare two fields in two different files and then print portions of each file on the same line when matched. It works reasonably well, but every now and again, I notice some errors and cannot seem to figure out what the issue may be and am turning to you for help. ... (2 Replies)
Discussion started by: jvoot
2 Replies

5. UNIX for Beginners Questions & Answers

How to count lines of CSV file where 2 fields match variables?

I'm trying to use awk to count the occurrences of two matching fields of a CSV file. For instance, for data that looks like this... Joe,Blue,Yes,No,High Mike,Blue,Yes,Yes,Low Joe,Red,No,No,Low Joe,Red,Yes,Yes,Low I've been trying to use code like this... countvar=`awk ' $2~/$color/... (4 Replies)
Discussion started by: nmoore2843
4 Replies

6. Programming

MYSQL - trigger to track changes to fields on update

So I have a php web application that allows updating of records. I would like to track the changes of only the fields that have changed. There are plenty of how to's that involve recreating the original row in a separate table with an additional time/date stamp column but I think this is a big... (1 Reply)
Discussion started by: barrydocks
1 Replies

7. Shell Programming and Scripting

awk help: Match data fields from 2 files & output results from both into 1 file

I need to take 2 input files and create 1 output based on matches from each file. I am looking to match field #1 in both files (Userid) and create an output file that will be a combination of fields from both file1 and file2 if there are any differences in the fields 2,3,4,5,or 6. Below is an... (5 Replies)
Discussion started by: ambroze
5 Replies

8. Shell Programming and Scripting

Source trigger in Unix shell

Hi All, Is ther any way to write the script with source trigger ? To trigger the script on particular time we have crontab. Similarly in a path, when a file comes particular script has to run. Thanks:) (7 Replies)
Discussion started by: jesu
7 Replies

9. Shell Programming and Scripting

The script wont trigger it. Thief Catcher program (open source)

me and my team working on a project called Stealth Hunter, Summarize about our project: Stealth Hunter is a thief catcher, It will silently take a snapshot of user using a stolen notebook or pc with webcam and send the information via email. how it work is: The scripts will triggered by the... (0 Replies)
Discussion started by: Kiraichi
0 Replies

10. UNIX for Advanced & Expert Users

Trimmng leading 0's from amount fields

Hi I need to remove leading zeros if that filed is amount filed and trimmimg leading and trailing spaces U | U|1999-01-01|000004384|000002192|0000000000000000.0000|0000000000000000.0000| U|XXX|0000000000000685.5800| FIXED |U Tried with this... (2 Replies)
Discussion started by: mohan705
2 Replies
Login or Register to Ask a Question