Shell Programming and Scripting

hi,
My file looks like this.

010,09102009,abc,30
010,09112009,abc,20
010,08102009,edf,40
010,09152009,edf,60
020,09102009,abc,30
020,09112009,abc,60

this continus like this , I need to add the last coloumn, based on the first coloumn and 3rd coloumn. If the first and third coloumn, are same in some other row then the last coloumn will add up. But the data is hugh 2 to 3 gb file so i cannot hard code the first and 3rd coloumn to add the last coloumn.
I want to check if the first and 3rd coloumn in some other rows are same then add up the last coloumn.
please suggest me a script.

like
if first row in the file above 010 and abc then add the last coloumn 30+20=50

If I understand your request correctly, this should do the trick:

Code:

awk -F, -v OFS="," '{a[$1OFS$3]+=$4}END{for(i in a) print i,a[i]}' file

Code:

awk -F, '{$NF=($1FS$3==v)?$NF FS ($NF+w):$NF;v=$1FS$3;w=$4}1' OFS="," file

perl:

Code:

my %hash;
open FH,"<a.txt";
while(<FH>){
  chomp;
  my @tmp = split(",",$_);
  my $key=$tmp[0].$tmp[2];
  $hash{$key}->{sum}+=$tmp[3];
  $hash{$key}->{val}=$hash{$key}->{val}.$_."\n";
}
foreach my $key(keys %hash){
 my @tmp=split("\n",$hash{$key}->{val});
 map {s/[0-9]+$/$hash{$key}->{sum}/}  @tmp;
 print join "\n", @tmp;
print "\n";
}

hi,
To the post of Ripat :

awk -F, -v OFS="," '{a[$1OFS$3]+=$4}END{for(i in a) print i,a[i]}' file

I am unable to understand this how $1OFS$3 is the index of the array. If the full command is explained then it will be helpful.

The solutions above are just guess work as you didn't post the required output.

{a[$1OFS$3]+=$4} for every line the array "a" indexed by "Col1,Col3" is increased by the value of Col4. OFS being the Output Field Separator, the comma in this case. For your first line, a[10,abc]==30. Second line a[10,abc]==50 etc...

END{for(i in a) print i,a[i]} when all lines have been processed, the array "a" is traversed and its index and value are printed.

hi Ripat,
This is working fine for me ....Thanks .
Actually I was not knowing tht awk array can take index as a string also.
Ur guess has hit the bull's eye. genius

---------- Post updated at 05:41 PM ---------- Previous update was at 05:41 PM ----------

hi Ripat,
This is working fine for me ....Thanks .
Actually I was not knowing tht awk array can take index as a string also.
Ur guess has hit the bull's eye. genius