Originally Posted by symphonic1985

Hey everyone,

So I have a task that I want to complete with awk (+ find, or something similar), but can't quite achieve it by myself...

I have 60 GB of files that I want to modify. They each consist of 2 columns of numbers, with up to 50,000 lines in a file.
e.g.

Code:

   1.607743       5.928237    
   1.578281       5.941602    
   1.480728       5.867404    
   1.592771       5.980338   
    ....               ....

The files are in subdirectories one layer down from the parent, and look like e.g.

Code:

322_b3.447/resu_1_1_0
322_b3.447/resu_2_1_0

I want to reduce the file sizes by averaging the numbers in the columns in groups of 10, and creating new files with names e.g. smaller/322_b3.447/resu_1_1_0

I've tried to make a do loop with general structure:

Code:

for file in $(find -type f -name 'resu_*')

Code:

do 
   some command to specify "$newfile"
   awk '
   {s1+=$1;s2+=$2}
   !(NR%10){
          {
       print (s1/10,s2/10) >  "$newfile"
     }
     s1=0; s2=0
   }' $file
done

However, I have trouble specifying the name of the new file because 'find' returns for $file names like ./resu_2_1_0, and I don't know how to remove the "./" and include a different directory there.

Also, my awk loop only works when the number of lines per file is exactly a multiple of 10. Otherwise the output for the remaining, say, 7 lines, will still be divided by 10 and hence be too small.