Hi all,

I have a file like the following:


ID,
2,Andrew,0,1,2,3,4,2,5,6,7,7,9,3,4,5,34,3,2,1,5,6,78,89,8,7,6......................
4,James,0,6,7,0,5,6,4,7,8,9,6,46,6,3,2,5,6,87,0,341,0,5,2,5,6....................
END,

(there are more entires on each line but to keep it simple I've left them off).




What I want to do is to sum every other value after the name on each line e.g. for Andrew I want to sum 0,2,4,5 etc then I want to sum the others e.g 1,3,2,6 etc and then print out the ID value, the name and the two totals.

e.g. 2,Andrew,164,133

I currently have the following:

$input_file3="$results_path/count_file.csv";
open(DAT3, $input_file3) || print "Could not open count file!";
@raw_data3=<DAT3>;
close(DAT3);

foreach $line (@raw_data3)
{
chop($line);
($VAR,$Name,$S1,$F1,$S2,$F2,$S3,$F3,$S4,$F4,$S5,$F5,$S6,$F6,$S7,$F7,$S8,$F8,$S9,$F9,$S10,$F10,$S11,$ F11,$S12,$F12,$S13,$F13,$S14,
$F14,$S15,$F15,$S16,$F16,$S17,$F17,$S18,$F18,$S19,$F19,$S20,$F20,$S21,$F21,$S22,$F22,$S23,$F23,$S24, $F24)=split(/,/,$line);

if ($VAR eq "ID" || $VAR eq "END")
{
`echo "ignoring this line"`
}
else
{
$suc = $S1 + $S2 + $S3 + $S4 + $S5 + $S6 + $S7 + $S8 + $S9 + $S10 +$S11 + $S12 + $S13 + $S14 + $S15 + $S16 + $S17 + $S18 + $S19
+ $S20 + $S21 + $S22 + $S23 + $S24;

$fail = $F1 + $F2 + $F3 + $F4 + $F5 +$F6 + $F7 + $F8 + $F9 + $F10 + $F11 + $F12 + $F13 + $F14 + $F15 + $F16 + $F17 + $F18 +$F19 +
$F20 + $F21 + $F22 +$F23 +$F24;

`echo "$CC,$Name,$suc,$fail" >> $tmp_path/suc_and_fail`;
}
}


The above works but it consumes a huge amount of memory and about 25% of my CPU for about 20 mins! The input files are quite big (approx 30,000 lines). Is there a more efficient way to do the above?

Thanks!