Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Processing of log files
Top Forums Shell Programming and Scripting Processing of log files Post 302949494 by scott_apc on Saturday 11th of July 2015 11:47:46 PM
Old 07-12-2015
Hi Don,

Basically, the requirement is to process the files in ascending date
Code:
2013.12.13

order, but descending suffix order
Code:
a.log

, and for every date and suffix, we need to check if the file exists and is greater than 0 size. Also, on failure, we need to output an appropriate message.

So far, I have tried this, but does give me expected sorted result, could someone point where I am doing the mistake :

Code:
for file in `cat file.txt | sort -k1,1nr -k2,2 -t _`; do echo $file; done

# Here is the file.txt :

Code:
ski2015.12.15.12.30.23.log
ski2015.12.15.12.30.24_a.log
ski2015.12.15.12.30.25_b.doc
ski2015.12.15.12.30.25_b.doc
ski2015.12.15.12.30.25_b.log
ski2015.12.15.12.30.25_b.log
ski2015.12.15.12.30.25_b.log
ski2015.12.14.12.30.24_a.log
ski2015.12.14.12.30.25_b.doc
ski2015.12.14.12.30.25_b.doc
ski2015.12.14.12.30.13_b.log
ski2015.12.14.12.29.25_b.log
ski2015.12.14.12.30.25_b.log
ski2015.12.14.12.30.23.log
ski2015.12.14.12.29.25_a.log
ski2015.12.15.12.30.23_a.log
ski2015.12.14.12.29.25_b.log
ski2015.12.14.12.29.25_c.log


Thanks.
Last edited by scott_apc; 07-12-2015 at 01:34 AM..
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Processing files within a directory one by one

Hi How to create a shell script which takes in to account all the files present within a directory DIR one by one e.g. suppose i have a directory named DIR where there are files with the extension .ABC i want to create shell script which processes all these files one by one. ... (1 Reply)
Discussion started by: skyineyes
1 Replies

2. UNIX for Advanced & Expert Users

FTP and Files processing

I want to do the following thing and I am newbie to Unix: 1) FTP files and save them in a folder 2) Pick the oldest file and also its name among the above FTPed files 3) Copy it to another folder X and also move the file to folder Y I know how to do the first task i.e., FTP the files and... (1 Reply)
Discussion started by: harish409
1 Replies

3. Shell Programming and Scripting

Two files processing

I have the following two files: 1st File: 1:1:100 2:101:400 3:401:450 4:451:600 5:601:980 6:981:1500 7:1501:1600 8:1601:1800 9:1801:2000 2nd File: 30 50 80 700 (2 Replies)
Discussion started by: moutaz1983
2 Replies

4. Shell Programming and Scripting

Processing files

Hi I have the folowing input file, the file looks like below sftp> . .. archive x001_ameint*.zip x001_ameint_1.zip x001_ameint_2.zip x001_REPORTS*.zip x001_REPORTS_1.zip sftp> I want my output to look like this x001_ameint*.zip x001_ameint_1.zip x001_ameint_2.zip (1 Reply)
Discussion started by: ramky79
1 Replies

5. Shell Programming and Scripting

Processing Multiple Files

Hello Everyone, I am new to scripting and confused with how to do this efficiently. I am trying to use AWK to do this. I have a lot of files in a folder which has the data of my throughput measurements in two columns i.e. Serial # and Throughput. like this 177.994 847.9 178.996 ... (1 Reply)
Discussion started by: hakim
1 Replies

6. Shell Programming and Scripting

get all files from a directory and pass the files for processing

Hi All, I have a directory in which there will be several files. i want to get all the files and pass it to a piece of code for processing on the files. This is the piece of code which does the processing. tr "\n" "|" < (log file name) | tr "$" "\n" > output echo ' ' >>output while... (1 Reply)
Discussion started by: suresh_kb211
1 Replies

7. UNIX and Linux Applications

Processing batches of files at a time

Hi I would like to run my script to process as many as 50 files at a time. Currently my script is being called like so: ./import.sh -f filename so I want to call my script in this way and it must execute every file in the directory /var/local/dsx/import (1 Reply)
Discussion started by: ladyAnne
1 Replies

8. Shell Programming and Scripting

Processing files using awk

Hi I have files in our UNIX directory like the below -rw-r--r-- 1 devinfo devsupp 872 Sep 14 02:09 IMGBTREE27309_12272_11_1_0_FK.idx0 -rw-r--r-- 1 devinfo devsupp 872 Sep 14 02:09 IMGBTREE27309_12272_11_0_0_PK.idx0 -rw-r--r-- 1 devinfo devsupp 432 Sep 14... (7 Replies)
Discussion started by: rbmuruga
7 Replies

9. Shell Programming and Scripting

script: processing files

Hi All, I have a log file which is attached here: ACCTXN.ALLOC, ACCTXN.IDX, ACCTXN.UDX, AE_COMS, AGENCY are different file names present in my input file (it could be 000s files and all names are starting with ). Now, I am interested in getting the files which are having " Predicted... (1 Reply)
Discussion started by: rishav
1 Replies

10. Shell Programming and Scripting

Processing multiple files

Hello I have a program cfxfrwb which is designed to remove headers from reports files. The cfxfrwb is located in the following directory /u01/efin/v40/live/bin I run the program against a single report file in the temp directory and it does it's job../cfxfrwb... (2 Replies)
Discussion started by: captainrhodes
2 Replies

LEARN ABOUT DEBIAN

grokevt-addlog

grokevt-addlog(1)														 grokevt-addlog(1)

NAME

       grokevt-addlog - A tool for adding a raw event log to an existing GrokEVT database.

SYNOPSIS

       grokevt-addlog  database-dir  evt-file new-type base-type .SH DESCRIPTION grokevt-addlog takes a raw event log (.evt file) and adds it to a
       pre-built database generated by grokevt-builddb(1). This new log file will be set up to use the message templates of another log, as deter-
       mined by the user.

       This  tool  is  primarily  useful for processing deleted logs and log fragments found on a system. While it is possible to use the database
       generated from one system with the logs of another, this is not recommended for investigations unless no alternatives exist.

ARGUMENTS

       grokevt-addlog uses the following arguments:

       database-dir
	      The base directory for the database generated previously by grokevt-builddb(1).

       evt-file
	      The file to be added to the database.

       new-type
	      The new log type/name that evt-file will take on.  This is the name that will need to be	used  later  with  grokevt-parselog(1)	to
	      access the new log. This type must not already exist in the database.

       base-type
	      The  existing  log  type that this new log will be based on. The message templates from this type will be used with the new log when
	      parsing. This type must exist in the current database.

BUGS

       Probably several. This particular script has not been extensively tested.

CREDITS

       Written by Timothy D. Morgan.

       Copyright (C) 2006-2007 Timothy D. Morgan

LICENSE

       Please see the file "LICENSE" included with this software distribution.

       This program is distributed in the hope that it will be useful, but WITHOUT ANY	WARRANTY;  without  even  the  implied	warranty  of  MER-
       CHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License version 2 for more details.

SEE ALSO

       grokevt(7) grokevt-builddb(1) grokevt-dumpmsgs(1) grokevt-findlogs(1) grokevt-parselog(1) grokevt-ripdll(1)

File Conversion Utilities					   20 March 2008						 grokevt-addlog(1)
All times are GMT -4. The time now is 12:49 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy