Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Match sum of values in each column with the corresponding column value present in trailer record
Top Forums UNIX for Dummies Questions & Answers Match sum of values in each column with the corresponding column value present in trailer record Post 302942298 by tpk on Monday 27th of April 2015 10:03:20 AM
Old 04-27-2015
Linux
Hi MadeInGermany,

Thank You for the solution!!!!.

The solution is working for pf_20150127.csv file, I ran the same script for pf_20150325.csv file and it got failed with the below error

Code:
Expected file(s) found, Performing Validations for file: pf_20150325.csv
pf_20150325.csv,20150325
------------------------------------------------------------------------------------
Checking Specific Validations 2 for File: pf_20150325.csv
------------------------------------------------------------------------------------
The sum of either or all columns is not matching with last row sum value of corresponding column. Hence exiting the Job
Errors: col 1: 173000000000 != 172928624441

So, I checked the temp files and found that the temp file temp_original_20150325.tmp where I am cutting the trailer record initially from original csv file, it is being read as below

Code:
cat temp_original_20150325.tmp

172928624441.00,334431290.00,346417133.00,354231936.00,443777494.00,526288959.00,769941370.00,918420217.00,1274200675.00,1067695005.00,1122762029.00,1181290201.00

And when I did a cat on the sum temp file temp_sum_20150325.tmp, it is calculated as below

Code:
cat temp_sum_20150325.tmp

1.73E+11,334431290,346417133,354231936,443777494,526288959,769941370,918420217,1274200675,1067695005,1122762029,1181290201

I checked in the csv file by opening it in excel and the value in the trailer record for column D is 1.72929E+11 and when I summed the rows under column D excluding Header and Trailer in the excel it turned out to be also same as that of trailer record which is 1.72929E+11. I don't understand why unix is reading the trailer record differently from the original file.

So, as there is difference in temp_original file and temp_sum file it's being failed. I don't understand why the original temp file is storing the values with XXXXXXXXXX.00. How can we make the code generic so that what ever value is present in the trailer record irrespective of e or E notation, my sum should be calculated accordingly. Please help me out.

With Regards,
TPK
Last edited by tpk; 04-27-2015 at 11:13 AM.. Reason: Correction
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to sum column 1 values

I have a file file like this. I want to sum all column 1 values. input A 2 A 3 A 4 B 4 B 2 Out put A 9 B 6 (3 Replies)
Discussion started by: suresh3566
3 Replies

2. Shell Programming and Scripting

print unique values of a column and sum up the corresponding values in next column

Hi All, I have a file which is having 3 columns as (string string integer) a b 1 x y 2 p k 5 y y 4 ..... ..... Question: I want get the unique value of column 2 in a sorted way(on column 2) and the sum of the 3rd column of the corresponding rows. e.g the above file should return the... (6 Replies)
Discussion started by: amigarus
6 Replies

3. Shell Programming and Scripting

Getting a sum of column values

I have a file in the following layout: 201008005946873001846130058030701006131840000000000000000000 201008006784994001154259058033001009527844000000000000000000 201008007323067002418095058034801002418095000000000000000000 201008007697126001722141058029101002214158000000000000000000... (2 Replies)
Discussion started by: jclanc8
2 Replies

4. Shell Programming and Scripting

Sum up the column values group by using some field

12-11-2012,PNL,158406 12-11-2012,RISK,4564 12-11-2012,VAR_1D,310101 12-11-2012,VAR_10D,310101 12-11-2012,CB,866 12-11-2012,STR_VAR_1D,298494 12-11-2012,STR_VAR_10D,309623 09-11-2012,PNL,1024106 09-11-2012,RISK,4565 09-11-2012,VAR_1D,317211 09-11-2012,VAR_10D,317211 09-11-2012,CB,985... (7 Replies)
Discussion started by: manas_ranjan
7 Replies

5. Shell Programming and Scripting

awk Print New Column For Every Two Lines and Match On Multiple Column Values to print another column

Hi, My input files is like this axis1 0 1 10 axis2 0 1 5 axis1 1 2 -4 axis2 2 3 -3 axis1 3 4 5 axis2 3 4 -1 axis1 4 5 -6 axis2 4 5 1 Now, these are my following tasks 1. Print a first column for every two rows that has the same value followed by a string. 2. Match on the... (3 Replies)
Discussion started by: jacobs.smith
3 Replies

6. Shell Programming and Scripting

Sum column values based in common identifier in 1st column.

Hi, I have a table to be imported for R as matrix or data.frame but I first need to edit it because I've got several lines with the same identifier (1st column), so I want to sum the each column (2nd -nth) of each identifier (1st column) The input is for example, after sorted: K00001 1 1 4 3... (8 Replies)
Discussion started by: sargotrons
8 Replies

7. Shell Programming and Scripting

Sum if line match with first column

Hi, i have log like below: A 2 5 B 4 1 C 6 8 B 0 1 C 1 0 B 2 3 A 0 0 i want to make result if match with A then sum from column 2 and 3 so the results: A 2 5 (5 Replies)
Discussion started by: justbow
5 Replies

8. Shell Programming and Scripting

Sum column values matching other field

this is part of a KT i am going thru. i am writing a script in bash shell, linux where i have 2 columns where 1st signifies the nth hour like 00, 01, 02...23 and 2nd the file size. sample data attached. Desired output is 3 columns which will give the nth hour, number of entries in nth hour and... (3 Replies)
Discussion started by: alpha_1
3 Replies

9. Shell Programming and Scripting

Help with calculate the total sum of record in column one

Input file: 101M 10M10D20M1I70M 10M10D39M4I48M 10M10D91M 10M10I13M2I7M1I58M 10M10I15M1D66M Output file: 101M 101 0 0 10M10D20M1I70M 100 1 10 10M10D39M4I48M 97 4 10 10M10D91M 101 0 10 10M10I13M2I7M1I58M 88 13 0 10M10I15M1D66M 91 10 1 I'm interested to count how many total of... (6 Replies)
Discussion started by: perl_beginner
6 Replies

10. UNIX for Beginners Questions & Answers

Sum the values in the column using date column

I have a file which need to be summed up using date column. I/P: 2017/01/01 a 10 2017/01/01 b 20 2017/01/01 c 40 2017/01/01 a 60 2017/01/01 b 50 2017/01/01 c 40 2017/01/01 a 20 2017/01/01 b 30 2017/01/01 c 40 2017/02/01 a 10 2017/02/01 b 20 2017/02/01 c 30 2017/02/01 a 10... (6 Replies)
Discussion started by: Booo
6 Replies

LEARN ABOUT CENTOS

processcsv.py

PROCESSCSV.PY(1)					      Virtualization Support						  PROCESSCSV.PY(1)

NAME

       processcsv.py - process virt-top CSV files

SUMMARY

	virt-top --csv data.csv
	processcsv.py < data.csv

DESCRIPTION

       virt-top is a top(1)-like utility for showing stats of virtualized domains.

       processcsv.py is a simple Python script that post-processes the output of "virt-top --csv".

       It is used like this:

	virt-top --csv data.csv
	processcsv.py < data.csv

       The second command will overwrite the following files in the current directory:

       "global.csv"
	   This contains the global (host) statistics columns from the CSV file.

       "domainNN.csv" (multiple files)
	   For each libvirt domain ID NN, a file is created containing the per-domain statistics from the CSV file.

SEE ALSO

       virt-top(1)

AUTHORS

       Richard W.M. Jones <rjones @ redhat . com>

COPYRIGHT

       (C) Copyright 2007-2012 Red Hat Inc., Richard W.M. Jones http://libvirt.org/

       This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by
       the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

       This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of
       MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.

       You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation,
       Inc., 675 Mass Ave, Cambridge, MA 02139, USA.

virt-top-1.0.8							    2013-12-29							  PROCESSCSV.PY(1)
All times are GMT -4. The time now is 01:37 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy