Sponsored Content
Full Discussion: Processing of log files
Top Forums Shell Programming and Scripting Processing of log files Post 302949496 by Don Cragun on Sunday 12th of July 2015 02:37:46 AM
Old 07-12-2015
This thread is titled "Processing of log files".

I don't want "Basically, the requirement is to process...". I gave you an explicit list of .log files in post #4 and asked what output you want when given that list of .log files.

Instead of answering that question you showed us an unsorted list of .log and .doc files (with some of those of files included in the list twice). Are .doc files now considered log files?

How did you get a list of files with duplicate names in that list? If you ran a find command to produce a combined list of files from multiple directories and stripped off the directory names, how do you expect a script to be able to guess at which directory a file came from to determine its size?

You have said you want to process the files with a suffix a.log and b.log and you imply that a file named ski2015.12.15.12.30.23.log doesn't have a suffix because there is no _ in that name. Please take the list I gave you or the list you gave us in post #5 and show us the EXACT output that you want to produce from that list (and explain why some files, if there are any, are not included in the output).

It is easy to see that a sort that sorts a list in reverse numeric order (when the field being sorted is not numeric) and for matching fields on that key sorts in increasing alphanumeric order on a second key doesn't give you a list in the order you have described. What is not easy is trying to guess what files are supposed to be excluded from your output and the order you do want for files that are not excluded if some of those files do not contain an _ character.

Why do you need a list of "log" files other than that produced by letting the shell expand *.log as a filename matching pattern in the directory where these files are located? Or, if you only want files with a suffix (as you have described it), why not just use *_*.log to get the list?
This User Gave Thanks to Don Cragun For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Processing files within a directory one by one

Hi How to create a shell script which takes in to account all the files present within a directory DIR one by one e.g. suppose i have a directory named DIR where there are files with the extension .ABC i want to create shell script which processes all these files one by one. ... (1 Reply)
Discussion started by: skyineyes
1 Replies

2. UNIX for Advanced & Expert Users

FTP and Files processing

I want to do the following thing and I am newbie to Unix: 1) FTP files and save them in a folder 2) Pick the oldest file and also its name among the above FTPed files 3) Copy it to another folder X and also move the file to folder Y I know how to do the first task i.e., FTP the files and... (1 Reply)
Discussion started by: harish409
1 Replies

3. Shell Programming and Scripting

Two files processing

I have the following two files: 1st File: 1:1:100 2:101:400 3:401:450 4:451:600 5:601:980 6:981:1500 7:1501:1600 8:1601:1800 9:1801:2000 2nd File: 30 50 80 700 (2 Replies)
Discussion started by: moutaz1983
2 Replies

4. Shell Programming and Scripting

Processing files

Hi I have the folowing input file, the file looks like below sftp> . .. archive x001_ameint*.zip x001_ameint_1.zip x001_ameint_2.zip x001_REPORTS*.zip x001_REPORTS_1.zip sftp> I want my output to look like this x001_ameint*.zip x001_ameint_1.zip x001_ameint_2.zip (1 Reply)
Discussion started by: ramky79
1 Replies

5. Shell Programming and Scripting

Processing Multiple Files

Hello Everyone, I am new to scripting and confused with how to do this efficiently. I am trying to use AWK to do this. I have a lot of files in a folder which has the data of my throughput measurements in two columns i.e. Serial # and Throughput. like this 177.994 847.9 178.996 ... (1 Reply)
Discussion started by: hakim
1 Replies

6. Shell Programming and Scripting

get all files from a directory and pass the files for processing

Hi All, I have a directory in which there will be several files. i want to get all the files and pass it to a piece of code for processing on the files. This is the piece of code which does the processing. tr "\n" "|" < (log file name) | tr "$" "\n" > output echo ' ' >>output while... (1 Reply)
Discussion started by: suresh_kb211
1 Replies

7. UNIX and Linux Applications

Processing batches of files at a time

Hi I would like to run my script to process as many as 50 files at a time. Currently my script is being called like so: ./import.sh -f filename so I want to call my script in this way and it must execute every file in the directory /var/local/dsx/import (1 Reply)
Discussion started by: ladyAnne
1 Replies

8. Shell Programming and Scripting

Processing files using awk

Hi I have files in our UNIX directory like the below -rw-r--r-- 1 devinfo devsupp 872 Sep 14 02:09 IMGBTREE27309_12272_11_1_0_FK.idx0 -rw-r--r-- 1 devinfo devsupp 872 Sep 14 02:09 IMGBTREE27309_12272_11_0_0_PK.idx0 -rw-r--r-- 1 devinfo devsupp 432 Sep 14... (7 Replies)
Discussion started by: rbmuruga
7 Replies

9. Shell Programming and Scripting

script: processing files

Hi All, I have a log file which is attached here: ACCTXN.ALLOC, ACCTXN.IDX, ACCTXN.UDX, AE_COMS, AGENCY are different file names present in my input file (it could be 000s files and all names are starting with ). Now, I am interested in getting the files which are having " Predicted... (1 Reply)
Discussion started by: rishav
1 Replies

10. Shell Programming and Scripting

Processing multiple files

Hello I have a program cfxfrwb which is designed to remove headers from reports files. The cfxfrwb is located in the following directory /u01/efin/v40/live/bin I run the program against a single report file in the temp directory and it does it's job../cfxfrwb... (2 Replies)
Discussion started by: captainrhodes
2 Replies
All times are GMT -4. The time now is 02:18 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy