Shell Programming and Scripting

Hi Team,

I'm looking for a command which removes files having 0 byte of having only header line (1 line).

My ETL process generates these files. Few files are not having header, in that case if no data from source, it will be 0 byte and few files are having header, in that case if no data from source, it will generate a byte with header information only, but no data.

I found the command to remove file with 0 byte

Code:

find . -type f -size 0 -exec rm {} \;

But not able to find command to remove file having only one line (header). I tried below command, but it is not working

Code:

find . -type f -size 0 -exec rm {} \;

I need it in form of command, as I do not want to create shell script to run at the end of ETL process

find them by time

Code:

 find . -ctime -1

1 is what was created for the last 24 hours

Hi digioleg54, I want to remove file which have 0 in size or have only one row.

Okay. You need to describe how to tell a header-only file from a file you want to keep.
If is simply size. Pretend example -

You want to delete 36 byte files and smaller, this finds files smaller than 37 and created in the last 24 hours:

Code:

find /path/to/files -type f -size -37c -ctime -1 -exec rm {} \;

One line files work this way:

Code:

find /path/to/files -type f  -ctime -1 |
while read fname 
do  
       sz=`cat $fname | wc -l`   # Not a UUOC done to get just a line count
       [ $sz -lt 2 ] && rm $fname
done

While it is true that:

Code:

wc -l "$fname"

pollutes the output with an unwanted filename:

Code:

cat $fname | wc -l

is still an unneeded use of cat. Try:

Code:

wc -l < "$fname"

instead.

The -ctime -1 was not required anywhere.


I'm afraid this requirement is vain; you should go for an adequate/applicable SOLUTION. Try this shell script

Code:

wc  -l * 2> /dev/null | while read CNT FN; do [ $CNT -ge 2 ] && [  "$FN" != total ] && echo rm "$FN"; done

I thought the goal here was to remove files with no lines or 1 line. Isn't the test above backwards. Shouldn't that be:

Code:

wc  -l * 2> /dev/null | while read CNT FN; do [ $CNT -lt 2 ] && [  "$FN" != total ] && echo rm "$FN"; done

and then rerun it without the echo shown in red above if the list looks right?