Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Help with solution to add together columns of large file

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Help with solution to add together columns of large file
# 1  
Old 09-03-2013
Help with solution to add together columns of large file
Hi everyone. I have a file with ~500 columns and I would like to perform a simple calculation on every two columns. The file looks like this:

Code:
$cat input
id   A   B   C   D   E   F.....X
1    2   4   2   3   4   1     n
2    4   6   4   6   4   5     n
3    4   7   5   2   2   3     n
4    5   3   9   1   2   8     n
5    6   2   2   8   1   7     n

I would like to perform the following calculation on every two columns:
Code:
id    A/(A +B + 100)    C/(C + D + 100)    E/(E + F + 100)....etc...
1            n    etc...
2            n
3            n
4            n
5            n

The resulting output file would therefore have the id's plus one new value for every two columns.

Any help is appreciated!!
Many thanks.
# 2  
Old 09-03-2013
Hint: Use awk (specifically gawk). You can make a formula there to calculate the intervening calc values based off the values every two columns, etc.

Is that enough of a hint?
# 3  
Old 09-03-2013
I know awk would be best for this, I haven't been able to figure it out. I'm quite the newb. It's the "every 2 columns" part that I don't know how to code.

I know how to do this:
Code:
awk -F "\t" '{print ($2/($3 + $2 + 100))}'

But I'm not sure how to do columns 2&3, 4&5, 6&7, 8&9

Many thanks for the help.
# 4  
Old 09-03-2013
You could try something like this:

Code:
awk -F "\t" '{
    printf "%s", $1 
    for(i=2; i<NF; i++) 
        printf "%s", OFS $i / ( $(i+1) + $i + 100 )
    printf "\n" }'

# 5  
Old 09-04-2013
This is very close to what I wanted, but it does this for every pair. I need it to do the first 2 columns (2&3) and then the next two columns after 2 and 3 (4&5).
Right now the code is doing 2$3, 3&4, 4&5, 5&6 etc... I know I can cut these columns out, but is there a way to code it differently? - 2&3, 4&5, 6&7, 8&9 etc...

Many thanks
# 6  
Old 09-04-2013
Code:
awk '
{
        # print id
        printf("%d ", $1);

        i=2
        # Add every 2 columns, output each val and sum
        while (i <= NF) {
                printf("%d %d %d ",$i,$(i+1),($i + $(i+1)));
                i+=2;
        }
        print "";
}'

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Extract spread columns from large file

Dear all, I want to extract around 300 columns from a very large file with almost 2million columns. There are no headers, but I can find out which column numbers I want. I know I can extract with the function 'cut -f2' for example just the second column but how do I do this for such a large... (1 Reply)
Discussion started by: fndijk
1 Replies

2. UNIX for Dummies Questions & Answers

Help with awk solution to add columns

Hi all. Wondering if someone can help with an awk solution to a problem I'm stumped with. I have a matrix file with >1000 fields and would like to add another column after each column with a text label. For example: Input: $cat file.txt name col1 col2 col3 coln aaaa ... (2 Replies)
Discussion started by: torchij
2 Replies

3. Shell Programming and Scripting

Any solution with awk for volatile columns??

Hi I have this file with content ale,4 ,ale,2 ,ale,1 ,ale,2 ale,1 ,ale,7 ,ale,7 ,ale,13 ale,6 ,ale,1 ,ale,1 ,ale,1 ale,1 ,ale,1 ,ale,37 ,ale,1 ale,1 ,ale,1 ,ale,2 ,ale,37 ale,77 ,ale,1 ,ale,53 ,ale,3 ale,5 ,ale,1 ,ale,2 ,ale,40 ale,1 ,ale,1 ,ale,44 ,ale,1... (7 Replies)
Discussion started by: nikhil jain
7 Replies

4. UNIX for Dummies Questions & Answers

Delete large number of columns rom file

Hi, I have a data file that contains 61 columns. I want to delete all the columns except columns, 3,6 and 8. The columns are tab de-limited. How would I achieve this on the terminal? Thanks (2 Replies)
Discussion started by: lost.identity
2 Replies

5. Shell Programming and Scripting

help printing two consecutive columns, every twenty in a large matrix

Hi, I'm having a problem printing two consecutive columns, as I iterate through a large matrix by twenty columns and I was looking for a solution. My input file looks something like this 1 id1 A1 A2 A3 A4 A5 A6....A20 A21 A22 A23....A4001 A4002 2 id2 B1 B2 B3 B4 B5 B6... 3 id3 ... 4 id4... (8 Replies)
Discussion started by: flotsam
8 Replies

6. Shell Programming and Scripting

Large file - columns into rows etc

I have done a couple of searches on this and have found many threads but I don't think I've found one that is useful to me - probably because I have very basic comprehension of perl and beginners shell so trying to manipulate a script already posted maybe beyond my capabilities.... Anyway - I... (26 Replies)
Discussion started by: Myrona
26 Replies

7. Shell Programming and Scripting

Large pipe delimited file that I need to add CR/LF every n fields

I have a large flat file with variable length fields that are pipe delimited. The file has no new line or CR/LF characters to indicate a new record. I need to parse the file and after some number of fields, I need to insert a CR/LF to start the next record. Input file ... (2 Replies)
Discussion started by: clintrpeterson
2 Replies

8. Shell Programming and Scripting

AWK solution to subtract multiple columns of numbers

Hope somebody is happy. NR==1 { num_columns=split( $0, menuend ); next; } { split( $0, substrend ); for ( i=1; i<=NF; i++ ) { minuend -= substrend; } } END { print "Result:"; for ( i=1; i<=num_columns; i++ ) { printf(... (3 Replies)
Discussion started by: awkward
3 Replies

9. Shell Programming and Scripting

Split large file and add header and footer to each small files

I have one large file, after every 200 line i have to split the file and the add header and footer to each small file? It is possible to add different header and footer to each file? (7 Replies)
Discussion started by: ashish4422
7 Replies

10. Shell Programming and Scripting

Split large file and add header and footer to each file

I have one large file, after every 200 line i have to split the file and the add header and footer to each small file? It is possible to add different header and footer to each file? (1 Reply)
Discussion started by: ashish4422
1 Replies
Login or Register to Ask a Question