Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

compare values in different lines of file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting compare values in different lines of file
# 1  
Old 10-08-2009
compare values in different lines of file
Hi everybody,

I have a file that looks like:

A B C D -1 0
E F G H -2 0
I J K L +1
M N O P -6

I would like to compare $5 of every line. If both values are negative, I calculate a mean value and write the first line and delete the second one. If the two $5 values are different only concerning plus or minus I want to leave them as they are. They same procedure when they are both positive values.

Could anyone help me with that problem???

Thank you so much,

Christine
# 2  
Old 10-08-2009
Not sure what exactly your output should look like, If you can give and example of the output that would be more easy to resolve.

From what i understood from your post
Chk this code

Code:
#!/usr/bin/perl

use strict;
use warnings;

my $prev_col_5_value = '';
my $prev_line ;
open my $fh , '<', 'abc.txt' || die $!;
while(<$fh>){
        chomp;
        my $line = $_;
        my @cols = split(/ /,$line);
        if ($prev_line){
                if ($prev_col_5_value < 0 && $cols[4] < 0 ) {
                        my $curr_col4_value = ($cols[4] + $prev_col_5_value )/2;
                        print "$prev_line\n";
                        $prev_line = '';
                        $prev_col_5_value = '';
                }
                else {
                        print "$prev_line\n";
                        $prev_line = $line;
                        $prev_col_5_value = $cols[4];
                        #print or dont print as per ur req

                }
        }
        else {
                $prev_line = $line;
                $prev_col_5_value = $cols[4];
        }
}
print "$prev_line \n" if ($prev_line);
close $fh;


Output

Code:
A B C D -1 0
I J K L +1
M N O P -6


Lemme know if this is not what it was meant to generate

Cheers,
Last edited by Franklin52; 10-09-2009 at 11:03 AM.. Reason: Please use code tags!
# 3  
Old 10-09-2009
Hi,

thank you for the fast help! This is pretty much, what the output should be. The only thing is the mean value. So, in the first line should be -1.5 in the 5th column (-1+(-2))/2=-1.5. Meaning, always, when both values are minus or plus, a mean value should be calculated.

So the output should look like:

A B C D -1.5 0
I J K L +1
M N O P -6

Could we do that?

Thanx again :-)
Christine

---------- Post updated at 09:46 AM ---------- Previous update was at 09:30 AM ----------

One more question: Would it be also possible with an awk script? I am much more familiar with awk than with perl ;-)

Thanx a lot,
Christine

---------- Post updated at 12:26 PM ---------- Previous update was at 09:46 AM ----------

Okay, I post my original file:

ATOM 0 BB SER 1 0 -31.958 -25.125 -11.061 1.00 0.00 -0.8
ATOM 1 BB GLY 1 1 -32.079 -26.085 -14.466 1.00 0.00 -0.4
ATOM 2 BB VAL 1 2 -36.455 -21.265 -15.792 1.00 0.00 4.2
ATOM 3 BB SER 1 3 -37.401 -20.877 -19.029 1.00 0.00 -0.8
ATOM 4 BB ALA 1 4 -42.701 -21.232 -18.584 1.00 0.00 1.8
ATOM 5 BB VAL 1 5 -47.498 -23.718 -18.979 1.00 0.00 4.2
ATOM 6 BB THR 1 6 -47.989 -24.426 -21.973 1.00 0.00 -0.7
ATOM 7 BB ALA 1 7 -46.376 -27.080 -22.868 1.00 0.00 1.8
ATOM 8 BB VAL 1 8 -44.852 -28.570 -20.796 1.00 0.00 4.2

If the values in the last column are both postive or both negative, then I want to calculate the mean value and write out only the line that originally contained the first value. If the values are different concerning plus/minus I leave them as they are. So in this case the output should be:

ATOM 0 BB SER 1 0 -31.958 -25.125 -11.061 1.00 0.00 -0.6
ATOM 2 BB VAL 1 2 -36.455 -21.265 -15.792 1.00 0.00 4.2
ATOM 3 BB SER 1 3 -37.401 -20.877 -19.029 1.00 0.00 -0.8
ATOM 4 BB ALA 1 4 -42.701 -21.232 -18.584 1.00 0.00 3.0
ATOM 6 BB THR 1 6 -47.989 -24.426 -21.973 1.00 0.00 -0.7
ATOM 7 BB ALA 1 7 -46.376 -27.080 -22.868 1.00 0.00 3.0

I think, the Perl script would work, I just cannot adapt it to my file :-)

I am really thankful for any help!
# 4  
Old 10-10-2009
Wrench
Hi Christine,

You could give this script a try. It runs with ksh:
Code:
#!/bin/ksh
prevline=""
while read line; do
  pval=${prevline##* }
  val=${line##* }
  if (( val*pval > 0 )); then
    avg=$(( (val+pval)/2 ))
    printf "%s %1.1f\n" "${prevline% *}" $avg
    line=""
  elif [[ $prevline != "" ]]; then
    echo $prevline
  fi
  prevline=$line
done < infile
if [[ $prevline != "" ]]; then
  echo $prevline
fi

Alternatively here is an awk equivalent
Code:
awk '{ if ( pval*$NF > 0 )
       { avg=(pval+$NF)/2
         sub(/ [^ ]*$/,"",prev)
         printf("%s %1.1f\n", prev, avg)
         $0=""
       }
       else if (prev != "")
         { print prev }
       sub(/ *$/,"")
       pval=prev=$0;
       sub(/.* /,"",pval)
     }
     END { if ( prev != "" )
           print prev
     }' infile

Last edited by Scrutinizer; 10-10-2009 at 05:18 PM..
This User Gave Thanks to Scrutinizer For This Post:
# 5  
Old 10-11-2009
Code:
#!/usr/bin/perl

use strict;
use warnings;

my $last_col = '';
my @new_arr;
open my $fh , '<', 'abc.txt' || die $!;
while(<$fh>){
        chomp;
        my @cols = split;
        if ($#new_arr > 0){
                if ($last_col < 0 && $cols[$#cols] < 0 ) {
                        #both -ve
                        my $curr_last_col_value = ($cols[$#cols] + $last_col )/2;
                        print join(' ',@new_arr,$curr_last_col_value)."\n";
                        @new_arr = ();
                        $last_col = '';
                }
                elsif ($last_col > 0 && $cols[$#cols] > 0){
                        # both +ve
                        my $curr_last_col_value = ($cols[$#cols] + $last_col )/2;
                        print join(' ',@new_arr,$curr_last_col_value)."\n";
                        @new_arr = ();
                        $last_col = '';
                }
                else {
                        #diff signs
                        #print or dont print as per ur req
                        print join(' ',@new_arr,$last_col) ."\n";
                        @new_arr = ();
                        push @new_arr, $cols[$_] for (0..$#cols-1);
                        $last_col = $cols[$#cols];
                }
        }
        else {
                push @new_arr, $cols[$_] for (0..$#cols-1);
                $last_col = $cols[$#cols];
        }
}
print join(' ',@new_arr,$last_col) if ($#new_arr > 0);
close $fh;

Ok this is the perl script which will do the job for your file format .
Just replace abc.txt with the file name you have. Alternatively if you want to run it for a sequence of files from command line , remove the open stmts and write while(<>) instead of while (<$fh>)
it could be run as cat file1 file2 | perl scriptname
Note:- I have added extra comments and self explanatory variable names for easy understanding

And reg the perl script replacing awk, I generally go with perl when ever there are more computations and the files tend to be large. However, its a matter of personal perception. Choose the one which suits you best.

Cheers,
Last edited by pludi; 10-12-2009 at 02:35 AM.. Reason: code tags please...
# 6  
Old 10-12-2009
Code:
open FH,"<a.txt";
my @tmp=<FH>;
for(my $i=0;$i<=$#tmp/2;$i++){
  my @t1=split(" ",$tmp[2*$i]);
  my @t2=split(" ",$tmp[2*$i+1]);
  if($t1[4]*$t2[4]>0){
    print join " ",(@t1[0..3],($t1[4]+$t2[4])/2,$t1[5]);
   print "\n";
  }
  else{
     print $tmp[2*$i];
     print $tmp[2*$i+1];
  }
}

# 7  
Old 10-12-2009
Thank you all very much!

Finally I used the Perl script and it worked perfectly!

You made my day :-)

Have a nice evening,
Christine
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Awk: compare values in two columns of the same file

I'm trying to learn awk, but I've hit a roadblock with this problem. I have a hierarchy stored in a file with 3 columns: id name parentID 4 D 2 2 B 1 3 C 1 1 A 5 I need to check if there are any values in column 3 that are not represented anywhere in column 1. I've tried this: awk '{arr;}... (7 Replies)
Discussion started by: kaktus
7 Replies

2. Shell Programming and Scripting

Compare Values in a Delimited Text file

Hi, How do I compare two columns within a text file If 2nd column values are same then I want to know 3rd column number matches or not Example: Prod Stag1 1234.79 Prod Stag2 1234.79 20 Prod Stag3 1234.79 30 Prod Stag4 1234.79 UAT Stag1 1243.56 UAT Stag2 1243.56 20 UAT ... (3 Replies)
Discussion started by: krux_rap
3 Replies

3. UNIX for Beginners Questions & Answers

Compare Values between column in the same file

Input File:- COLUMN1 COLUMN2 COLUMN3 COLUMN4 COLUMN5 COLUMN6 SMS Email AO Mail Post N Cell VEGE Potato E W 396 12 0 384 0 0 0 0 0 VEGE Onion S W 17 0 17 0 0 0 0 0 0 FRUIT APPLE N W 549 61 0 0 0 0 0 488 0 FRUIT APPLE SE W 291 14 239 38 0 10 0 0 0 FRUIT APPLE EAMS W 397 32 309 56 309 309 0... (27 Replies)
Discussion started by: Nina2910
27 Replies

4. Shell Programming and Scripting

How to compare the values of a column in a same file using awk?

Dear Unix experts, I have got a file where I would like to compare the values of second column if first column is same in such a way that the difference between the values is >50. If not, I would like to discard both values. For example, my input file looks like - comp275_c0_seq2 73... (7 Replies)
Discussion started by: utritala
7 Replies

5. Homework & Coursework Questions

Compare to values in a file in unix

Here is sample file ===============Index 0=================== isActive=0, Input=1, Output=1, Status=1 State = Future , PRIMARY UnderCount=2 inCount=2 outCount=0 SCount=673 -- ===============Index 1=================== isActive=0, Input=1, Output=1, Status=1 ... (1 Reply)
Discussion started by: sooda
1 Replies

6. Shell Programming and Scripting

Compare values in two files. For matching rows print corresponding values from File 1 in File2.

- I have two files (File 1 and File 2) and the contents of the files are mentioned below. - I am trying to compare the values of Column1 of File1 with Column1 of File2. If a match is found, print the corresponding value from Column2 of File1 in Column5 of File2. - I tried to modify and use... (10 Replies)
Discussion started by: Santoshbn
10 Replies

7. Shell Programming and Scripting

How to compare the values of a column in awk in a same file and consecutive lines..

I would like to compare the values of 2nd column of consecutive lines of same file in such a way so that if the difference between first value and second value is more than 100 it should print complete line else ignore line. Input File ========== PDB 2500 RTDB 123 RTDB-EAGLE 122 VSCCP 2565... (4 Replies)
Discussion started by: manuswami
4 Replies

8. Shell Programming and Scripting

I need to extract last column of a file and compare the values

Hi, I am new to unix and I need help in solving below mentioned issue, really appreciate ur help. I have a file sam, john, 2324, 07142007 tom, thomson, 2343, 07142007 john, scott, 2478, 07142007 its a comma delimited file, I need to extract the last column from each line and this... (4 Replies)
Discussion started by: vukkusila
4 Replies
Login or Register to Ask a Question