Shell Programming and Scripting

Hi,

I have typical logs file something of these formats -

Code:

fn2013.12.13.log
fn2013.12.13_a.log
fn2013.12.13_b.log

suffix part is after the underscore ( ie a.log or b.log )

I need to process the files in ascending date order, but descending suffix order, and check if the file exist and is greater than zero, and on failure an appropriate message should be displayed like "file is of zero size".

Coming from just a normal Unix knowledge background, it is difficult for me to process the file based on the sorting logic as described above. Could someone help me on this. ?

I will really appreciate if someone could help me this or guide me through.

Thanks.

Moderator's Comments:

Please use CODE tags for all sample input, sample output, and code segments.

How would above three sample files be sorted? Where would the no-suffix file fn2013.12.13.log end up?

Thanks for the reply!

Well if it does have the suffix, it will be just sorted out of date - the first parameter. I need to pull date (2013.12.13) and suffix (a.log or b.log) -- if present, from the file names.

That doesn't answer the question... If you have the following files:

Code:

fn2013.12.13.log
fn2013.12.13_a.log
fn2013.12.13_b.log
fn2013.12.14.log
fn2013.12.14_a.log
fn2013.12.15.log
fn2013.12.16.log
fn2013.12.17.log
fn2013.12.17_a.log
fn2013.12.17_b.log
fn2013.12.17_c.log

exactly what are you trying to do with these files? And, exactly what output(s) are you trying to produce with this set of files?

Hi Don,

Basically, the requirement is to process the files in ascending date

Code:

2013.12.13

order, but descending suffix order

Code:

a.log

, and for every date and suffix, we need to check if the file exists and is greater than 0 size. Also, on failure, we need to output an appropriate message.

So far, I have tried this, but does give me expected sorted result, could someone point where I am doing the mistake :

Code:

for file in `cat file.txt | sort -k1,1nr -k2,2 -t _`; do echo $file; done

# Here is the file.txt :

Code:

ski2015.12.15.12.30.23.log
ski2015.12.15.12.30.24_a.log
ski2015.12.15.12.30.25_b.doc
ski2015.12.15.12.30.25_b.doc
ski2015.12.15.12.30.25_b.log
ski2015.12.15.12.30.25_b.log
ski2015.12.15.12.30.25_b.log
ski2015.12.14.12.30.24_a.log
ski2015.12.14.12.30.25_b.doc
ski2015.12.14.12.30.25_b.doc
ski2015.12.14.12.30.13_b.log
ski2015.12.14.12.29.25_b.log
ski2015.12.14.12.30.25_b.log
ski2015.12.14.12.30.23.log
ski2015.12.14.12.29.25_a.log
ski2015.12.15.12.30.23_a.log
ski2015.12.14.12.29.25_b.log
ski2015.12.14.12.29.25_c.log

Thanks.

This thread is titled "Processing of log files".

I don't want "Basically, the requirement is to process...". I gave you an explicit list of .log files in post #4 and asked what output you want when given that list of .log files.

Instead of answering that question you showed us an unsorted list of .log and .doc files (with some of those of files included in the list twice). Are .doc files now considered log files?

How did you get a list of files with duplicate names in that list? If you ran a find command to produce a combined list of files from multiple directories and stripped off the directory names, how do you expect a script to be able to guess at which directory a file came from to determine its size?

You have said you want to process the files with a suffix a.log and b.log and you imply that a file named ski2015.12.15.12.30.23.log doesn't have a suffix because there is no _ in that name. Please take the list I gave you or the list you gave us in post #5 and show us the EXACT output that you want to produce from that list (and explain why some files, if there are any, are not included in the output).

It is easy to see that a sort that sorts a list in reverse numeric order (when the field being sorted is not numeric) and for matching fields on that key sorts in increasing alphanumeric order on a second key doesn't give you a list in the order you have described. What is not easy is trying to guess what files are supposed to be excluded from your output and the order you do want for files that are not excluded if some of those files do not contain an _ character.

Why do you need a list of "log" files other than that produced by letting the shell expand *.log as a filename matching pattern in the directory where these files are located? Or, if you only want files with a suffix (as you have described it), why not just use *_*.log to get the list?

The structure of the entries in your file.txt doesn't fit those in post#1, and it doesn't represent dates. It vaguely reminds me of date/time entries; should the time be considered in the sort?

Please be way more specific and precise; post a sample output that you need!