UNIX for Dummies Questions & Answers

Hello Everyone,

I have to subsample a large text file - 17,021,811 lines. I need to keep every other block of 290 lines (keep first 290 lines, throw away the next 290, keep the next 290, etc.)

So, I wrote the following bash script:

Code:

 
i=0
start=1
stop=290
LIMIT=29449
 
while [ "$i" -lt "$LIMIT" ]
 
do
 
sed -n "$start , $stop p" file_src >> file_dest
 
i=$(($i+1))
start=$((1+2*290+$i))
stop=$(($start+289))
 
done

So, this works but it is taking forever and I am wondering if there is a better/more efficient way.

Thank you very much in advance for any advice.

try:

Code:

awk ' BEGIN {i=1}
      { if(FNR%290==0) { i=!i}
        if(i) {print $0} } '  bigfile > subsample

Thank you very much! It is much appreciated.