Grep Logs That Are Updating


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Grep Logs That Are Updating
# 1  
Old 01-18-2013
Grep Logs That Are Updating

Hello all. I am new to this forum and also very new to using grep - so please excuse me if this question is not in the correct forum and/or is not pertinent to this website.

I use greps like this:

Code:
gunzip -c L:\System1\SailLogger_20071019* L:\System2\SailLogger_20071019* L:\System3\SailLogger_20071019* | grep 35=Order

These logs are updating continuosly throughout the day. SailLogger_20071019* is the name of log followed by YYYYMMDD. I use the wildcard to target all the logs (the logs themselves are renamed at the top of each hour - eg 2007101907 (for 7am) and 2007101908 (for 9am).

When I run this grep, it will only extract from the first log in the statement - System1. I then get an error message related to not being able to find the end of the file.

Is there a way to grep all these logs ?

Thanks you for any help and guidance.

Regards.

Moderator's Comments:
Mod Comment edit by bakunin: Please use CODE-tags. It is in the rules you should have read. We even have this code tag video for how to use code tags when posting code and data.

Last edited by bakunin; 01-30-2013 at 11:40 AM..
# 2  
Old 01-18-2013
It is very strange and unusual for a gzipped log to be continually appended to. Are you sure all of these logs are already compressed?
# 3  
Old 01-18-2013
Hi Thanks for replying. Yes these logs are updating every 30 seconds. I tried this grep -

gunzip -c L:\System1\SailLogger_20130118* L:\System2\SailLogger_20130118* L:\System3\S
ailLogger_20130118* | grep 35=SailErrorNotice

and got results for System1 logs but then received this message -


gunzip: L:\System1\SailLogger_2013011813.tlg.gz: unexpected end of file

And in case it matters, I am running these greps in a windows environment.
# 4  
Old 01-18-2013
I just noticed that you're unzipping the same thing three times in a row:

Code:
L:\System1\SailLogger_20130118*
L:\System2\SailLogger_20130118*
L:\System3\SailLogger_20130118*

Is that really intentional? Didn't you want three different logfiles?
# 5  
Old 01-18-2013
Hi. Yes. Each path has a differnt log (System1,2,3) that follows the same naming convention.
# 6  
Old 01-18-2013
Do you know about the zgrep command (cf. man zgrep):
Quote:
NAME
zgrep - search possibly compressed files for a regular expression
# 7  
Old 01-18-2013
Pardon my blind eyes, I don't know how I missed that.

Anyway. If this code is for a real shell and not windows cmd, I'd try this:

Code:
for FILE in L:\System1\SailLogger_20130118* L:\System2\SailLogger_20130118* L:\System3\SailLogger_20130118*
do
        gunzip < "$FILE"
done | grep 35=SailErrorNotice

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Grep last 5 mins from logs

Hi, system date format Thu Jun 13 12:55:18 EDT 2019 My log date format 09.148.192.60 - - "GET /akamai/sureroute-test-object.html HTTP/1.1" 404 231 can someone please help me, how to get last 5mins of logs please ? I need the command Please wrap your samples/codes in CODE TAGS,... (3 Replies)
Discussion started by: scazed
3 Replies

2. Shell Programming and Scripting

Search string or words in logs without using Grep

I'm in need of some kind of script that will search for a string in each logfile in a directory but we don't want to use GREP. GREP seems to use up to much of our memory causing the server to use up a lot of swap space. Our log files are bigger than 500M on a daily basis. We lately started... (8 Replies)
Discussion started by: senormarquez
8 Replies

3. Shell Programming and Scripting

Grep a pattern in current date logs

Hello, I need to write one script which should search particular pattern like ABCD in log file name hello.txt only in current date logs. in current directory i have so many past date logs but grep should be applied on current date logs. on daily basis current date logs are in number 30 and... (2 Replies)
Discussion started by: ajju
2 Replies

4. Shell Programming and Scripting

Script to grep logs for Errors

Hi Guys, I want to write a script which can grep the logs (server.log) from a file for Error String and output to a other file. Problems: How to know about the errors only between the current restart and not in previous as server.log has earlier restarts also? thanks for the help! Much... (5 Replies)
Discussion started by: ankur328
5 Replies

5. Shell Programming and Scripting

How to check whether logs are updating or not?

how to check whether logs are updating or not in unix is there any built in command or function ? (1 Reply)
Discussion started by: mail2sant
1 Replies

6. Shell Programming and Scripting

how to grep the logs for two particular timestamp

Hi, could anyone help me out how to write a script, to grep the two timestamp from a particular file, so that it will list out all the logs between the particular timestamp I have a pattern of log: servicename operationname starttime endtime eg., servicename1 operationname1 01:11:11... (1 Reply)
Discussion started by: jacktolearn
1 Replies

7. Shell Programming and Scripting

Grep for lines in web logs

I want to find the unique url in a apache logs which got 404 error . I can do something like "cat apache.log|grep 404|awk '{print $2,$3}'|grep 404 this will give me say /foo.html 404 /foo.html 404 /foo.html 404 /bar.html 404 /cat.html 404 However my output should only find... (3 Replies)
Discussion started by: gubbu
3 Replies

8. Shell Programming and Scripting

Grep yesterday logs from weblogic logs

Hi, I am trying to write a script which would go search and get the info from the logs based on yesterday timestamp and write yesterday logs in new file. The log file format is as follows: """"""""""""""""""""""""""... (3 Replies)
Discussion started by: harish.parker
3 Replies

9. Shell Programming and Scripting

Grep logs on the fly

Hi, We use an application that is dumping logs to a file on disk. However, this is dumping very verbosely and there is no method of turning down the logging level. We need to remove certain contents from these before they are commited to disk. Has anybody got any ideas how I can do this... (3 Replies)
Discussion started by: harperonline
3 Replies

10. UNIX for Dummies Questions & Answers

Constantly updating log files (tail -f? grep? awk?)

I have a log file which is continuously added to, called log.file. I'd like to monitor this file, and when certain lines are found, update some totals in another file. I've played around with tail -f, grep, and awk, but can't seem to hit the right note, so to speak. The lines I'm... (0 Replies)
Discussion started by: nortonloaf
0 Replies
Login or Register to Ask a Question