UNIX for Advanced & Expert Users

Maybe you should consider the use of a RAM disk to store these files containing huge data that are daily updated...

You could feed the files twice, like so:

Code:

awk -f  test.awk test*.txt test*.txt

Then you can test if the second round of files is coming like so for example:

Code:

FILENAME==ARGV[1] && NR>FNR{secondround=1}
secondround{print "Making SD calculations"}

ARGV[1] contains the first filename. If the current filename FILENAME equals ARGV[1] for the second time, you will have processed all the files once...
This way you would need to store a lot less information

------------
Or you can do it like this, setting the variable half-way. Then you do not have to test inside, which might be a speed advantage:

Code:

awk -f  test.awk test*.txt secondround=1 test*.txt

and inside the awk script:

Code:

secondround{print "Making SD calculations"}

Yup, your first proposal

Code:

FILENAME==ARGV[1] && NR>1{secondround=1} secondround{print "Making SD calculations"}

doesn't work since as soon as NR>1 (even if we are in the first round, it will setup secondround to 1)

But the following should do the trick

Code:

awk '!f[FILENAME":"FNR]++{
print "firstround";
next}
{ 
print "second round here"
}' tst* tst*

That's right, I meant to write: FILENAME==ARGV[1] && NR>FNR. Anyway I much prefer my second option, it is cleanest.
I think your suggestion would mean a lot of extra array elements, no? Why not leave out ":"FNR and use a variable?

@Scruti,

Yeah i agree that setting the variable directly in the ARGV line is the best option, i guess we just cross-posted, so that when i wrote my previous post i didn't see the last update of yours (#9)

Actually I updated my post after you pointed out the mistake ( see edit comment ) ...

Thanks both of you guys. This discussion has been really useful specially this latest round of comments from you both Scrutinizer and ctsgnb. I did not know that you could actually pass awk variable in the middle of the files. Okay this maybe worth a try.
Thanks again for this.