Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Subtracting columns against each other

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Subtracting columns against each other
# 1  
Old 02-25-2010
Subtracting columns against each other
Hi All,

I have a file of 100 lines of each having 1000 columns. I need to find the difference of each column against each other. That means, Col1-Col1; Col1-Col2; Col1-Col3;......Col1-Col1000; Col2-Col1; Col2-Col2; Col2-Col3;.... and so on ....up to Col1000-Col1000.

Lets say the file is having 5 lines of each having 5 columns. Input files as follows:
Code:
            Col1    Col2    Col3   Col4   Col5
Line1       A        B         C      D       E
Line2       A        B         C      D       E
Line3       A        B         C      D       E
Line4       A        B         C      D       E
Line5       A        B         C      D       E

The output I am expecting is as follows:
Code:
            Col1  Col2   Col3  Col4   Col5
Line1       0     A-B    A-C   A-D    A-E
Line2       0     A-B    A-C   A-D    A-E
Line3       0     A-B    A-C   A-D    A-E
Line4       0     A-B    A-C   A-D    A-E
Line5       0     A-B    A-C   A-D    A-E

Line6     B-A       0    A-C   A-D    A-E
Line7     B-A       0    A-C   A-D    A-E
Line8     B-A       0    A-C   A-D    A-E
Line9     B-A       0    A-C   A-D    A-E
Line10    B-A       0    A-C   A-D    A-E

.
.
.
.
.
.
.
.
Line25    E-A     E-B    E-C   E-D      0

For this i have used the following code

HTML Code:
awk '{for(i=1; i<NF; i++) {for(j=1; j<NF; j++) {s=s FS $j-$i} print s;s=""}}{print "\n"}' infile > outfile
But I am not getting the result as i expected. I got the following result:
Code:
           Col1    Col2     Col3  Col4   Col5
 Line1       0     A-B      A-C   A-D    A-E
Line2      B-A       0      B-C   B-D    B-E
Line3      C-A     C-B        0   C-D    C-E
Line4      D-A     D-B      D-C     0    D-E
Line5      E-A     E-B      E-C   E-D      0

Line6        0     A-B      A-C   A-D    A-E
Line7      B-A       0      A-C   A-D    A-E
.
.
.
.
.
.
up to
.
.
Line25     E-A     E-B      E-C   E-D      0

But this is not the expected form. So, I need help from anyone to sort-out this problem.

Can anyone help me in this regard? Expecting your reply and thanks in advance.

Warm regards
Fredrick.

---------- Post updated at 04:58 PM ---------- Previous update was at 04:42 PM ----------

The above problem in a simplified way:

Inputfile:

1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1
1 1 1 1 1

Code:

Code:
awk '{for(i=1; i<NF; i++) {for(j=1; j<NF; j++) {s=s FS $j-$i} print s;s=""}}{print "\n"}' infile > outfile

Got the following output:
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0


0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0


0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0


0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0


0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0


Can anyone help me in this regard?

Thanks in advance.

Fredrick.

---------- Post updated at 05:18 PM ---------- Previous update was at 04:58 PM ----------

Since the above example input file is having the same values in all the columns, its not good to have that example.

I used the following as one more example:

Input file:
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5

Code:

Code:
awk '{for(i=1; i<NF; i++) {for(j=1; j<NF; j++) {s=s FS $j-$i} print s;s=""}}{print "\n"}' infile > outfile

Got the following as output:
0 1 2 3
-1 0 1 2
-2 -1 0 1
-3 -2 -1 0


0 1 2 3
-1 0 1 2
-2 -1 0 1
-3 -2 -1 0


0 1 2 3
-1 0 1 2
-2 -1 0 1
-3 -2 -1 0


0 1 2 3
-1 0 1 2
-2 -1 0 1
-3 -2 -1 0


0 1 2 3
-1 0 1 2
-2 -1 0 1
-3 -2 -1 0


Can anyone help me in this regard?

Warm regards
Fredrick.
Last edited by Scott; 02-25-2010 at 11:54 AM.. Reason: Added more code tags...
# 2  
Old 02-25-2010
You do know that with your requirements you'll get n^2 columns of output for any n columns of input (5 columns input -> 25 cols output, 1.000 input -> 1.000.000 output)

That aside, try if this fits you:
Code:
#!/usr/bin/perl -W

use strict;
use warnings;

while ( my $line = <DATA> ) {
    my @cols = split / /, $line;
    my ( $l, $r ) = ( 0, 0 );
    for ( $l = 0 ; $l <= $#cols ; $l++ ) {
        for ( $r = 0 ; $r <= $#cols ; $r++ ) {
            print $cols[$l] - $cols[$r], " ";
        }
    }
    print "\n";
}

__DATA__
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5
1 2 3 4 5

Exemplary output:
Code:
$ perl cols.pl
0 -1 -2 -3 -4 1 0 -1 -2 -3 2 1 0 -1 -2 3 2 1 0 -1 4 3 2 1 0 
0 -1 -2 -3 -4 1 0 -1 -2 -3 2 1 0 -1 -2 3 2 1 0 -1 4 3 2 1 0 
0 -1 -2 -3 -4 1 0 -1 -2 -3 2 1 0 -1 -2 3 2 1 0 -1 4 3 2 1 0 
0 -1 -2 -3 -4 1 0 -1 -2 -3 2 1 0 -1 -2 3 2 1 0 -1 4 3 2 1 0 
0 -1 -2 -3 -4 1 0 -1 -2 -3 2 1 0 -1 -2 3 2 1 0 -1 4 3 2 1 0

Last edited by pludi; 02-25-2010 at 12:50 PM.. Reason: typo
# 3  
Old 03-01-2010
Hi Pludi,

Thank you very much for your reply. I have tried the following code:

HTML Code:
#!/usr/bin/perl -W

use strict;
use warnings;

while ( my $line = <DATA> ) {
    my @cols = split / /, $line;
    my ( $l, $r ) = ( 0, 0 );
    for ( $l = 0 ; $l <= $#cols ; $l++ ) {
        for ( $r = 0 ; $r <= $#cols ; $r++ ) {
            print $cols[$l] - $cols[$r], " ";
        }
    }
    print "\n";
}
While executing the file, i am getting the error message as follows:

HTML Code:
Name "main::DATA" used only once: possible typo at ./example.pl line 6.
readline() on unopened filehandle DATA at ./example.pl line 6.
Can you tell me, where is the mistake? As per the error message, in line 6 DATA has been used only once.

Expecting your reply and thanks in advance.

Warm regards
Fredrick.
# 4  
Old 03-01-2010
Code:
#!/usr/bin/perl -W

use strict;
use warnings;
open(DATA,"< infile.txt") ;
while ( my $line = <DATA> ) {
    my @cols = split / /, $line;
    my ( $l, $r ) = ( 0, 0 );
    for ( $l = 0 ; $l <= $#cols ; $l++ ) {
        for ( $r = 0 ; $r <= $#cols ; $r++ ) {
            print $cols[$l] - $cols[$r], " ";
        }
    }
    print "\n";
}

# 5  
Old 03-01-2010
Quote:
Originally Posted by Fredrick
While executing the file, i am getting the error message as follows:

HTML Code:
Name "main::DATA" used only once: possible typo at ./example.pl line 6.
readline() on unopened filehandle DATA at ./example.pl line 6.
Can you tell me, where is the mistake? As per the error message, in line 6 DATA has been used only once.
I've been using the DATA pseudo-file (which is the whole section after the __DATA__ line in my original code) in lieu of a real file. You'll have to open() that yourself, or use <> instead of <DATA> if you're reading from stdin anyways.
# 6  
Old 03-02-2010
Thank you ahmad.diab and pludi., its working fine.

warm regards
Fredrick.

---------- Post updated at 01:36 PM ---------- Previous update was at 09:57 AM ----------

Hi All,

To continue with the same problem, I would like to get the output in the following manner:

Code:
0 -1 -2 -3 -4 
0 -1 -2 -3 -4
0 -1 -2 -3 -4
0 -1 -2 -3 -4
0 -1 -2 -3 -4
1 0 -1 -2 -3  
1 0 -1 -2 -3  
1 0 -1 -2 -3  
1 0 -1 -2 -3 
1 0 -1 -2 -3 
2 1 0 -1 -2 
2 1 0 -1 -2 
2 1 0 -1 -2 
2 1 0 -1 -2 
2 1 0 -1 -2 
3 2 1 0 -1 
3 2 1 0 -1 
3 2 1 0 -1 
3 2 1 0 -1 
3 2 1 0 -1 
4 3 2 1 0 
4 3 2 1 0 
4 3 2 1 0 
4 3 2 1 0 
4 3 2 1 0

can anyone help me in this regard?

Expecting your reply and thanks in advance.

Warm regards
Fredrick.
# 7  
Old 03-02-2010
I have to say, this starts to sound more and more like homework. That, or you're just plain lazy and expect us to do your work.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk - Adding and Subtracting Numbers from 2 Columns

Hi Folks, I have a file with 2 columns TAB delimited and I want to add '1' to the first column and subtract '-1' from the second column. What I have tried so far is; awk -F"\t" '{ $1-=1;$2+=1}1' OFS='\t' file File 0623 0623 0624 0624 0643 0643 1059 1037 1037 1037 1038 1038... (2 Replies)
Discussion started by: pshields1984
2 Replies

2. Answers to Frequently Asked Questions

Subtracting two files

Hi, I want to subtract 2 files and save the remaining text in another file. Lets say, Hello Happy // Hi * Hungry File2 Happy Hi Output Hello (5 Replies)
Discussion started by: beginner_99
5 Replies

3. Shell Programming and Scripting

Subtracting values from variable

Legends, Please help me in , how do i subtract the variable values listed like below. the first value of orig should be subtracted from first value of prev and so on. san> echo $orig 346 316 340 239 410 107 291 139 128 230 167 147 159 159 172 116 110 260 177 0 177 169 168 186 165 366 195... (15 Replies)
Discussion started by: sdosanjh
15 Replies

4. Shell Programming and Scripting

Searching columns and subtracting values in awk

Hi everyone, I had a similar question a couple days ago but my problem has gotten significantly (to me anyway) more complex. I have two files: File 1: 0808 166 166 62 9 0 1000fights 1 1 2 1 0 100places2visit 2 2 2 2 0 10veronica91 167 167 3 1 0 11thgorgeous 346 346 3806 1461 122... (2 Replies)
Discussion started by: collards
2 Replies

5. UNIX for Dummies Questions & Answers

Subtracting values from 2 columns in a file

Hello, I have a file with 5 columns that looks like this: A1BG chr19 + 58863335 58866549 A1BG chr19 - 58858171 58864865 A2LD1 chr13 - 101182417 101186056 A2LD1 chr13 - 101182417 101241046 A2M chr12 - 9220303 9268558 A2ML1 ... (5 Replies)
Discussion started by: wolf_blue
5 Replies

6. Shell Programming and Scripting

Subtracting with awk?

i have a small awk script which prints the 5 columns of different o/p i want the 5th column subtracted from 100 and then display the result .. but i do not get the desired result .. I 'm using following script awk ' BEGIN { FS="" RS="us" } { ... (3 Replies)
Discussion started by: fugitive
3 Replies

7. Shell Programming and Scripting

subtracting variables in ksh

hi all, how do i subract variables in shell ?? am trying to space out the headers and the output generated by the shell so they all line up : currently the output is like this : servers : users server1 : 10 latestServer : 50 so i thought... (3 Replies)
Discussion started by: cesarNZ
3 Replies

8. Shell Programming and Scripting

comparing files - adding/subtracting/formating columns

I have two files: file1.txt: FS Total Used Free Used% /u01 10000 8000 2000 80% /u02 10000 8000 2000 80% /u03 10000 8000 2000 80% /u04 10000 8000 2000 80% /u05 10000 8000 2000 80% /u06 10000 8000 2000 80% /u07 10000 8000 2000 80% /u10 10000 5000 5000 50% file2.txt:... (7 Replies)
Discussion started by: oabdalla
7 Replies

9. UNIX for Dummies Questions & Answers

Subtracting an Integer from a Variable

Hello, I am in following situation.- COUNT=`ls -l | wc -l` echo $COUNT ---> 26 NO_OF_FILES=$COUNT-1 echo $NO_OF_FILES ---> 26-1 Here, I want the output to be 25. How could I do this. It seems simple, but I am not getting it. Please help me. (2 Replies)
Discussion started by: The Observer
2 Replies

10. Shell Programming and Scripting

Subtracting date / timestamps

I have looked through the forums and found many date / time manipulation tools, but cannot seem to find something that fits my needs for the following. I have a log file with date time stamps like this: Jun 21 17:21:52 Jun 21 17:24:56 Jun 21 17:27:59 Jun 21 17:31:03 Jun 21 17:34:07 Jun... (0 Replies)
Discussion started by: roadcyclist
0 Replies
Login or Register to Ask a Question