10-23-2018
Need Optimization shell/awk script to aggreagte (sum) for all the columns of Huge data file
Optimization shell/awk script to aggregate (sum) for all the columns of Huge data file
File delimiter "|"
Need to have Sum of all columns, with column number : aggregation (summation) for each column
File not having the header
Like below -
Column 1 "Total
Column 2 : "Total
...
...
Column N : "Total :
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Friends,
I have to write a shell script,the description is----
i Have to check the uniqueness of the numbers in a file.
A file is containing 200thousand tickets and a ticket have 15 numbers in asecending order.And there is a strip that is having 6 tickets that means 90 numbers.I... (7 Replies)
Discussion started by: namishtiwari
7 Replies
2. Shell Programming and Scripting
Hi I'm new to this forum and I'm a beginner when it comes to shell and awk programming. But I have the following problem:
I have 5 csv files (data1.csv, data2.csv, etc.) and need to calculate the average between the total sum of the 1st and 7 column.
csv example:... (3 Replies)
Discussion started by: sapo51
3 Replies
3. Shell Programming and Scripting
can anyone help me how do i add the colums using awk seperated by character @. for eg i have
3@4
2@9
5@1
the result should be
10 14
i tried using
{ sum+= $1 }
END { print sum }
but it just gives the result 10. can anyone help me with this one
thank you and best regards (7 Replies)
Discussion started by: phone_book
7 Replies
4. UNIX for Advanced & Expert Users
Hi Experts,
I got a question..
In the following output of `ps -elf | grep DataFlow` I get:-
242001 A mqsiadm 2076676 1691742 0 60 20 26ad4f400 130164 * May 09 - 3:02 DataFlowEngine EAIDVBR1_BROKER 5e453de8-2001-0000-0080-fd142b9ce8cb VIPS_INQ1 0
242001 A mqsiadm... (5 Replies)
Discussion started by: varungupta
5 Replies
5. Shell Programming and Scripting
Hi All,
Need an urgent help to convert a unix file in to a particular format:
**source file:**
1111111
2d2f2h2
3dfgsd3
...........
1111111 <-- repeats in every nth line. remaining all lines will be different
123ss41
432ff45
...........
1111111 <-- repetition
qwe1234
123weq3... (1 Reply)
Discussion started by: rajivnairfis
1 Replies
6. Shell Programming and Scripting
I am new to shell script.I have records like below in 4 different files which have about 10000 records each, all records unique and sorted based on column 2.
1 2 3 4 5 6
---------------------------
SR|1010478|000044590|1|0|0|
SR|1014759|000105790|1|0|0|
SR|1016609|000108901|1|0|0|... (2 Replies)
Discussion started by: reach.sree@gmai
2 Replies
7. Shell Programming and Scripting
Hi All,
I have some data like below.
Step1,Param1,Param2,Param3
1,2,3,4
2,3,4,5
2,4,5,6
3,0,1,2
3,0,0,0
3,2,1,3
........
so on
Where I need to find the median(arithmetic) of each column from Param1...to..Param3 for each set of Step1 values.
(Sort each specific column, if the... (5 Replies)
Discussion started by: ks_reddy
5 Replies
8. Shell Programming and Scripting
Hi All,
I need the modification for the below mentioned code (found in one more post https://www.unix.com/shell-programming-scripting/27161-script-generate-average-values.html) to find the average values for all the columns(but for a specific rows) and print the averages side by side.
I have... (4 Replies)
Discussion started by: ks_reddy
4 Replies
9. Shell Programming and Scripting
Dear all ,
I found that if we work with thousands line of data, awk does not work perfectly. It will cut hundreds line (others are deleted) and works only on the remain data.
I used this command : awk '$1==1{$1="Si"}{print>FILENAME}' coba.xyz to change value of first column whose value is 1... (4 Replies)
Discussion started by: ariesto
4 Replies
10. Shell Programming and Scripting
Hello everyone
I am a beginner in Shell scripting. Need your help to achieve desired result.
I have a file (sample format below)
001g8aX0007jxLz xxxxxxxxxxxxxxx 9213974926411 CO-COMM-133 CO-L001-DLY 7769995578239 44938 1 1
... (1 Reply)
Discussion started by: Rohit Mallah
1 Replies
COLRM(1) BSD General Commands Manual COLRM(1)
NAME
colrm -- remove columns from a file
SYNOPSIS
colrm [start [stop]]
DESCRIPTION
The colrm utility removes selected columns from the lines of a file. A column is defined as a single character in a line. Input is read
from the standard input. Output is written to the standard output.
If only the start column is specified, columns numbered less than the start column will be written. If both start and stop columns are spec-
ified, columns numbered less than the start column or greater than the stop column will be written. Column numbering starts with one, not
zero.
Tab characters increment the column count to the next multiple of eight. Backspace characters decrement the column count by one.
ENVIRONMENT
The LANG, LC_ALL and LC_CTYPE environment variables affect the execution of colrm as described in environ(7).
EXIT STATUS
The colrm utility exits 0 on success, and >0 if an error occurs.
SEE ALSO
awk(1), column(1), cut(1), paste(1)
HISTORY
The colrm command appeared in 3.0BSD.
BSD
August 4, 2004 BSD