Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Taking a specific value from a log file
Top Forums Shell Programming and Scripting Taking a specific value from a log file Post 302780999 by RudiC on Friday 15th of March 2013 10:55:19 AM
Old 03-15-2013
Quote:
Originally Posted by Lord Spectre
. . . Btw, I believe the second one is faster, am I wrong? I'm asking that because the gclog sometime becomes huge!
That would be an interesting question. awk will need to go through the entire file to the end to be sure it picks the last number wanted. With tac, you could take the first one and quit. BUT - tac still needs to read the entire file to find the last, before last etc. lines and present them to sed.

And, if there's many a non matching trailing line, this may become slower and slower.

Can you time the solutions with a couple of input files and report back?

EDIT: I created a huge file and did the timing:
Code:
$ time   awk 'match ($0, "->") {gsub (/^.*>|[^0-9].*$/, ""); TMP=$0} END{print TMP}' file 
3101902

real    0m0.081s
user    0m0.072s
sys     0m0.008s
$ time tac file | sed '/-/!d;s/.*>//;s/K.*//;q'
3101902

real    0m0.008s
user    0m0.000s
sys     0m0.000s

Very evident, one order of magnitude difference in execution times. In fact, with strace you can see, that awk opens the file and reads reads reads
Code:
open("file", O_RDONLY)                  = 3   
ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fffeff21e78) = -1 ENOTTY (Inappropriate ioctl for device)
read(3, "185413.854: [GC 3938735K->310031"..., 4096) = 4096 
read(3, "570 secs]\n185472.378: [GC 336997"..., 4044) = 4044
read(3, "185490.946: [GC 3940426K->310132"..., 4096) = 4096 
read(3, "185513.458: [GC 3940411K->310190"..., 4096) = 4096

, while tac does an lseek (... SEEK_END), (... SEEK_SET) and starts from the rear:
Code:
open("file", O_RDONLY)                  = 3
lseek(3, 0, SEEK_END)                   = 5495040
. . . 
lseek(3, 5488640, SEEK_SET)             = 5488640
read(3, "14570 secs]\n185472.378: [GC 3369"..., 6400) = 6400
. . .

EDIT 2:
Code:
tac file | awk 'match ($0, "->") {gsub (/^.*>|[^0-9].*$/, ""); print; exit}'

is as fast!
Last edited by RudiC; 03-15-2013 at 12:24 PM..
This User Gave Thanks to RudiC For This Post:
Lord Spectre
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

read specific text from a log file

Hi guys I need to retrieve the values in BOLD that I have mentioned in the below log file. I want to store those values in a variable, preferably the same name as the column name in the log file. if you paste the below mentioned log file in a notepad and remove the word wrap.. u will get a... (4 Replies)
Discussion started by: ragha81
4 Replies

2. Shell Programming and Scripting

How to read a specific value from a Log file?

Hi, I have a .log file in which it has many values. But i need some specific values. How it can be done using Shell Script. Please explain in detail. Thankx in advance. Sathish D V. (8 Replies)
Discussion started by: cooolthud
8 Replies

3. Shell Programming and Scripting

Extract Specific pattern - log file

Hello everyone, I am on AIX (6.1). I can only use shell (ksh) script. I can't do this on my own, so will do my best to explain my needs.I also do not know what is the best idea to make it work, so here is what I am thinking, but I may wrong. I need help to extract info on... (3 Replies)
Discussion started by: Aswex
3 Replies

4. Shell Programming and Scripting

Write out specific data from log to a new file

I got a huge log in zipped files, i need to write out lines by specific data and if the line with the same contains XML message with the same sessionID will be written to the file to. The log structure: 2013-08-16 16:31:06,810 ( 122: rogate) INFO - UId:10453, GId:5422: new... (16 Replies)
Discussion started by: batka
16 Replies

5. UNIX for Dummies Questions & Answers

Search for a specific String in a log file for a specific date range

Hi, I have log file which rolls out every second which is as this. HttpGenRequest - -<!--OXi dbPublish--> <created="2014-03-24 23:45:37" lastMsgId="" requestTime="0.0333"> <response request="getOutcomeDetails" code="114" message="Request found no matching data" debug="" provider="undefined"/>... (3 Replies)
Discussion started by: karthikprakash
3 Replies

6. Shell Programming and Scripting

Need specific columns in a log file as excel.

Hi All... I am in need of few columns from a log file.. in .xls file... below is what i have tried. my log file has 16 colums with " ; " as delimiter, but i need randomn columns 1 2 3 4 5 6 10 11 16 in an excel. I tried to awk the columns with delimiter ; and it worked, below is the log... (5 Replies)
Discussion started by: nanz143
5 Replies

7. Shell Programming and Scripting

Check specific content from log file

Hi all, i have a logfile which is continuously being updated. I have built a script to check for a specific content and if it is found, it sends a string into a file. Here's the current script: #!/bin/bash logfile=/opt/jboss-eap-6.3/standalone/log/server.log tail -fn0 $logfile | \... (7 Replies)
Discussion started by: nms
7 Replies

8. UNIX for Beginners Questions & Answers

Viewing a specific timeframe of a log file

Hi guys Done a bit of research online but can't seem to figure it out, is there anyway of grepping or using sed to view a specific time period of a log file. I am trying to view a log file for Saturday 22nd April between 08:00 - 12:00 I saw this command online and tried but doesn't seem to... (10 Replies)
Discussion started by: simpsa27
10 Replies

9. UNIX for Beginners Questions & Answers

Selecting specific variable in log file

Hi there I am trying to look for a specific word in the log file and I am aware this can be done by grep for example. As there will be multiple entries for this I want to grep the last one to enter the log... how would I go about this - would I have to use tail? Thanks in advance Alex (4 Replies)
Discussion started by: simpsa27
4 Replies

10. Shell Programming and Scripting

Grep a log file starting from a specific time to the end of file

I have a log file which have a date and time at the start of every line. I need to search the log file starting from a specific time to the end of file. For example: Starting point: July 29 2018 21:00:00 End point : end of file My concern is what if the pattern of `July 29 2018 21:00:00`... (3 Replies)
Discussion started by: erin00
3 Replies
All times are GMT -4. The time now is 09:03 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy