Shell Programming and Scripting

I am trying to delete all the files with line number less than 100 in a directory. Thank you.

Well, first, the obligatory question: is this a homework question? If so, we can't help you, have a read of the:

https://www.unix.com/unix-dummies-que...om-forums.html

Assuming that it's not, what is the real world problem of your question?

Regards

Thank you for your reply. This is not a homework question. I need to send a lot of files to an R program to calculate some features for each file. But since R is so slow and files with line numbers less than 100 is meaningless anyway, I decided to erase those unqualifed and send the rest to the R program. Thank you very much.

One way -

Code:

#!/bin/ksh
find /path/to/files -type f |\
while read file
do
     wc -l | read linecount junk
     if [[ $linecount -lt 100 ] ; then
           rm -f $file
     fi
done

Make sure you have a good backup first.

To develop this piecemeal, here's a simple way to print the line count of each file:

Code:

wc -l *

To print only those files which have fewer than 100 lines, use grep or awk to remove those with a line count over 100:

Code:

wc -l * | awk '$1 < 100'

(The default action of awk is to print anything matching the condition. We are using awk instead of grep because of developments below.)

To actually delete those files instead of just print them, extend the awk script to only print the file name (presuming there will be no file names with whitespace in them):

Code:

wc -l * | awk '$1 < 100 { print $2 }' | xargs rm

You might want to try the intermediate stages before deploying this, just to verify that it works for you.

You could use awk or Perl all the way if you have a lot of large files; you'd obviously only really need to read the first 100 lines to see if a file is large enough.

Thanks so much.