Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

List last 10 Hrs file and find error files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting List last 10 Hrs file and find error files
# 1  
Old 02-07-2010
Question List last 10 Hrs file and find error files
Hi,

I need a script that can search a word "Error" in last 10 Hrs generated logs in /log/App1 and /log/App2 folder..

Note these directories have massive log files ...actually our application generate 100 Log files of size 2MB in just a min so script must be fast enough to cater this I see lot of post on the net and try different things but unable to find any yet .. All Help is requested
# 2  
Old 02-07-2010
What is the format of the log file contents and log file naming, for instance is there a log file per hour and does each log file entry have the date and time in it? What Unix are you using?

If there is a new log file per hour (which would make sense based on the amount of log data being generated then you can simply find all files modified in the last 10 hours and grep then for "Error", e.g.:
Code:
find /log/App1 /log/App2 -type f -cmin -600 -exec grep Error {} \;

The options to find vary on the Unix your are using, if the version you are using only provide the -mtime option then you may have to specify any file modified in the last day instead and use grep to filter on log file entries time stamps.
[/code]
# 3  
Old 02-08-2010
Hi,

Your script raise below error , Using AIX
Code:
find . -type f -cmin -600 -exec grep Error {} \;

 ** Error [ 202 ] **
jsh t24asu ~ --> ** Error [ STDIN ] **
Error getting input from STDIN , errno = 19

Below is the sample format of logs
Code:
"_BNK/SYSTEM.END.OF.DAY5_REVALUATION.AL_7_30 JAN 2010_13:26:49_SELECT F.JOB.LIST.12 SAMPLE 100000 Selected=36 tim
092981 _BNK/SYSTEM.END.OF.DAY5_REVALUATION.AL_7_30 JAN 2010_13:26:49_Processing...36 Offset...13 Number of sessions...1
092982 ..... ** Error [ EXIT_TRANS_ABORT ] **
092983 TRANSACTION ABORT forced as transaction open when program terminated
092984 jsh t24asu ~ -->.[K.[J.jsh t24asu ~ -->tSA 7.[K.[J.......[21C
092985 tSA 7
092986 Error: COMO is already running on this port
092987 Agent 7 started 30 JAN 10 13:28:51
092988 Running on server acbs04
092989
092990  1 Records selected
092991
092992 _BNK/SYSTEM.END.OF.DAY5_REVALUATION.AL_7_30 JAN 2010_13:28:54_SELECT F.COMPANY WITH @ID = BA0010001 Selected=1 t
092993
092994  1062 Records selected
092995
092996 _BNK/SYSTEM.END.OF.DAY5_REVALUATION.AL_7_30 JAN 2010_13:28:55_SSELECT F.BATCH BY BATCH.STAGE Selected=1062 time=
092997 _BNK/SYSTEM.END.OF.DAY5_REVALUATION.AL_7_30 JAN 2010_13:28:57_Job Status from Disk  2
092998 _BNK/SYSTEM.END.OF.DAY5_REVALUATION.AL_7_30 JAN 2010_13:28:57_Job Status from Disk  2 "

log files are not overlapping every service generate a seperate log per min for its service activity details
Last edited by Scott; 02-08-2010 at 09:57 AM.. Reason: Please use code tags
# 4  
Old 02-08-2010
I have never come across jsh, so perhaps you can try the suggestion on Bourne (sh), Korn (ksh) or bash shells?

Okay the contents of the log file does not have the time stamp in the same line as the error so something cleverer than grep will be required, so do you get a new log file every hour or one per day or what?
Last edited by TonyFullerMalv; 02-08-2010 at 05:31 PM..
# 5  
Old 02-09-2010
Hi,
New Log file created after every 1 min and there are more than 100 services which create the log means more than 100 log files per minute.

---------- Post updated at 12:59 AM ---------- Previous update was at 12:55 AM ----------

HI

Still wont work in ksh and bsh
Code:
$ bsh
abcd04:$PWD
$ find . -type f -cmin -600 -exec grep Error {} \;
abcd04:$PWD
$

Last edited by Scott; 02-09-2010 at 03:54 AM.. Reason: Code tags, PLEASE!
# 6  
Old 02-09-2010
For bash one would type:
Code:
bash

Unless AIX aliases bsh to bash?

Any AIX experts out there who can fix a simple find statement for us?
# 7  
Old 02-09-2010
do you have time stamp in the log file name?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to find list of missing files based on the file format?

Hi All, In the file names we have dates. Based on the file format given by the user, if any file is not existed for a particular date with in a given interval we should consider that file is missing. I have the below files in the directory /bin/daily/voda_files. ... (9 Replies)
Discussion started by: nalu
9 Replies

2. Shell Programming and Scripting

Find command to find a word from list of files

I need to find a word '% Retail by State' in the folder /usr/sas/reports/RetailSalesTaxallocation. When I tried like below, -bash-4.1$ cd /usr/sas/reports/RetailSalesTaxallocation -bash-4.1$ find ./ -name % Retail by State find: paths must precede expression: Retail Usage: find ... (10 Replies)
Discussion started by: Ram Kumar_BE
10 Replies

3. Shell Programming and Scripting

finding files N hrs old

I know you can supply the find command with an option to find files > than N days old. Is there some way to do this to find files that are > than N hours old. I want to do somthing like this: find . -mtime + (now - 2hrs) -print If not is there a way to do this with sed or awk or some... (5 Replies)
Discussion started by: BeefStu
5 Replies

4. Shell Programming and Scripting

To delete files older than 24 hrs

I have to retain only 1 day files in my system an I have to delete all the other files which are older than 24 hrs. Please let me know the option I have to give in the find -mtime command. (3 Replies)
Discussion started by: rajesh8s
3 Replies

5. Shell Programming and Scripting

Want to find the stdout for the partiuclar user login for past 12 hrs.

Want to find the stdout for the partiuclar user login for past 12 hrs. Say for eg : user login id is teladm And the host name is sys22prod I want to see the stdout for that user id in that host for past 12 hrs (1 Reply)
Discussion started by: mail2sant
1 Replies

6. Shell Programming and Scripting

find list of files from a list and copy to a directory

I will be very grateful if someone can help me with bash shell script that does the following: I have a list of filenames: A01_155716 A05_155780 A07_155812 A09_155844 A11_155876 that are kept in different sub directories within my current directory. I want to find these files and copy... (3 Replies)
Discussion started by: manishabh
3 Replies

7. Shell Programming and Scripting

Find files in a directory which are older than 2 hrs

hi all, I need to find files in a directory which are older than N hrs... n can be 1,2,3,.. etc when tried using -mtime option it gives all the files in the last 24hrs from the current time . please help me out on this .. thanks (8 Replies)
Discussion started by: sparks
8 Replies

8. UNIX for Dummies Questions & Answers

How to list files Created in last 2 hrs

I wanted to know what command should I use to see the files created in last 2 hours in a given directory. I know to see the files changed in last one day I can use this: find /admin//dump -type f -ctime -1 -print | xargs ls -lt|pg However I am not getting what should I use if I wanted tol... (4 Replies)
Discussion started by: rsonakiya
4 Replies

9. Shell Programming and Scripting

check for new file over 24 hrs

Hi all, I am trying to figure out a method to flag an error if a file has not arrived in a certain directory within 24 hrs of the last one arriving. I am currently writing the time to a file when a file is recieved. I then want to check for a new file reguarly but flag an error if it has been... (3 Replies)
Discussion started by: pxy2d1
3 Replies

10. Shell Programming and Scripting

modifying grep to get files only within last 2 hrs

Hi gurus I am currently using the below mentioned grep to find timestamp of last generated log file. touch -t $time_search dummy ecust_time_stamp=$(find . -name 'eCustomerCME*' -newer dummy -type f -exec ls -ltr {} \; | tail -1 | awk ' { print $6,$7,$8 } ') I calculate... (3 Replies)
Discussion started by: ragha81
3 Replies
Login or Register to Ask a Question