Need Optimization shell/awk script to aggreagte (sum) for all the columns of Huge data file


Login or Register to Reply

 
Thread Tools Search this Thread
# 1  
Need Optimization shell/awk script to aggreagte (sum) for all the columns of Huge data file

Optimization shell/awk script to aggregate (sum) for all the columns of Huge data file

File delimiter "|"
Need to have Sum of all columns, with column number : aggregation (summation) for each column
File not having the header

Like below -
Column 1 "Total
Column 2 : "Total
...
...
Column N : "Total :
# 3  
something along these lines:
Code:
awk -F'|' '{for(i=1;i<=NF;i++) a[i]+=$i} END{for(i=1; i in a;i++) printf("Column %d: %d\n", i, a[i])}' myFile

These 2 Users Gave Thanks to vgersh99 For This Post:
Login or Register to Reply

|
Thread Tools Search this Thread
Search this Thread:
Advanced Search

More UNIX and Linux Forum Topics You Might Find Helpful
awk based script to find the average of all the columns in a data file
ks_reddy
Hi All, I need the modification for the below mentioned code (found in one more post https://www.unix.com/shell-programming-scripting/27161-script-generate-average-values.html) to find the average values for all the columns(but for a specific rows) and print the averages side by side. I have...... Shell Programming and Scripting
4
Shell Programming and Scripting
Awk based script to find the median of all individual columns in a data file
ks_reddy
Hi All, I have some data like below. Step1,Param1,Param2,Param3 1,2,3,4 2,3,4,5 2,4,5,6 3,0,1,2 3,0,0,0 3,2,1,3 ........ so on Where I need to find the median(arithmetic) of each column from Param1...to..Param3 for each set of Step1 values. (Sort each specific column, if the...... Shell Programming and Scripting
5
Shell Programming and Scripting
Sum up values of columns in 4 files using shell script
reach.sree@gmai
I am new to shell script.I have records like below in 4 different files which have about 10000 records each, all records unique and sorted based on column 2. 1 2 3 4 5 6 --------------------------- SR|1010478|000044590|1|0|0| SR|1014759|000105790|1|0|0| SR|1016609|000108901|1|0|0|...... Shell Programming and Scripting
2
Shell Programming and Scripting
A variable and sum of its value in a huge data.
varungupta
Hi Experts, I got a question.. In the following output of `ps -elf | grep DataFlow` I get:- 242001 A mqsiadm 2076676 1691742 0 60 20 26ad4f400 130164 * May 09 - 3:02 DataFlowEngine EAIDVBR1_BROKER 5e453de8-2001-0000-0080-fd142b9ce8cb VIPS_INQ1 0 242001 A mqsiadm...... UNIX for Advanced & Expert Users
5
UNIX for Advanced & Expert Users
Shell script to check the unique numbers in huge data
namishtiwari
Friends, I have to write a shell script,the description is---- i Have to check the uniqueness of the numbers in a file. A file is containing 200thousand tickets and a ticket have 15 numbers in asecending order.And there is a strip that is having 6 tickets that means 90 numbers.I...... Shell Programming and Scripting
7
Shell Programming and Scripting