Shell Programming and Scripting

You could also try:-

Code:

Orig_IFS="$IFS"
IFS=\~
sort Tax_Provision_Sample.dat | while read p1 p2 p3 p4 p5 p6 p7 p8 p9 p10
do
   echo "$p2~$p4~$p3~$p8~$p9"
done > $fn.txt
IFS="$Orig_IFS"

I'm not great with awk, so this is simpler to read, but there may well be a trade off on performance.


Just another option, although you may already have better.



Robin

Robin, where does fn get set and shouldn't that be changed inside the loop?

I was just using the output file name from the originally supplied code. I cannot say how it was originally set.

What do you think of the performance issues for a large input? Would my code by horribly slower? If so, then the original poster must make the decision of speed over clarity (assuming that my suggestion is clear, and I'm not sure if it is)



Regards,
Robin

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Hang on, no I have totally misread the supplied attempt.

If I re-read it, the code is generating multiple output files based on the input records and writing the whole line to the appropriate file.

No, forget my suggestion, totally wrong.

I can't think of a way to remove the "open/append, write, close" operations to write to multiple files unless we force is another way by working out what files there could possibly be and then getting the records for each required output file in turn. That would just generate more headaches than it solves and for a large input file could still be quite slow.


Am am a fool



Robin

Don't we all have that from time to time

In shell the open/append, write, close operations are performed implicitly by the scope of the redirection of the file descriptor. RudiC already gave a suggestion using an array.

With you multi-variable approach:

Code:

sort Tax_Provision_Sample.dat | 
while IFS="~" read p1 p2 p3 p4 p5 p6 p7 p8 p9 p10
do
  printf "%s\n" "$p2~$p4~$p3~$p8~$p9" >> $fn.txt
done

The files would need to be empty beforehand..

And where does fn get set now?
I think you mean

Code:

#!/bin/bash
declare -A AF
sort Tax_Provision_Sample.dat | 
while IFS="~" read p1 p2 p3 p4 p5 p6 p7 p8 p9 p10
do
  fn="$p2~$p4~$p3~$p8~$p9"
  if [ -z "${AF[$fn]}" ]; then
    > $fn.txt
    AF[$fn]=1
  fi
  echo "$p1~$p2~$p3~$p4~$p5~$p6~$p7~$p8~$p9~$p10" >> $fn.txt
done

I have added some bash-4 code that will empty the output files when first met.
Omit the code if you have bash-3 (and delete/empty the files before you run the script).

Yes something like that . Was trying to show open/append, write, close operations in shell .. and forgot about $fn ...

Oh how wonderful:-

Code:

while IFS="~" read .......

Lovely code! That certainly neatens some of the rubbish I've written. I hope you don't mind if I steal it......?



Robin
(Still learning)