get last section from large logfile


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting get last section from large logfile
# 1  
Old 05-19-2004
get last section from large logfile

Hi all,

I need to pull the (last section) from the header to the end of file. This logfile will have many sections appended, but I only need to capture the last one and email it to someone. Any ideas? Thanks Kathy

Example Logfile:


=============
= BOX : SAPCFI
= JOB : SAPCFI
= Commande : -D -k A -C cfg_PK pr
=
= Wed May 19 13:10:25 EST 2004
=
blah
blah
blah


=============
= BOX : SAPCFI
= JOB : SAPCFI
= Commande : -D -k A -C cfg_PK pr
=
= Wed May 19 13:21:25 EST 2004
=
blah
blah
blah
# 2  
Old 05-19-2004
This will do the trick - using awk

Code:
#!/bin/sh

awk 'BEGIN { FS="\n"; 
RS="" }
{
     record=$0
}
END { print record }' your_log_file

exit 0

Cheers
ZB
http://www.zazzybob.com
# 3  
Old 05-19-2004
ZB,

That was close. I don't even understand the code.
But it returned the last 3 lines of the file. I needed probably about 60 lines back.

I think it picked up from the last blank line to the end. Let me describe my format again. These lines are varied with blank line in between various multiple lines.

Example Logfile:

=============
= BOX : SAPCFI
= JOB : SAPCFI
= Commande : -D -k A -C cfg_PK pr
=
= Wed May 19 13:10:25 EST 2004
=
blah
blah
blah
blank line
blah
blah
blank line
blah
blank line


=============
= BOX : SAPCFI
= JOB : SAPCFI
= Commande : -D -k A -C cfg_PK pr
=
= Wed May 19 13:21:25 EST 2004
=
blah
blah
blah
blank line
blah
blah
blank line
blah
blank line
# 4  
Old 05-19-2004
Change the RS value from RS="" to RS="===*\n" The RS value is the Record Separator value. FS is field separator. Changing RS to "===*\n" specifies that 1 record is delimited by multiple === values with a new line at the end of the record. Each field within one of these records is delimited by FS, in this case a new line.

this could be shortened to
cat yourfile | awk 'BEGIN{RS="===*\n";FS="\n";} END{print $NR}'

Last edited by google; 05-19-2004 at 06:09 PM..
# 5  
Old 05-19-2004
I tried the cat version and my screen went blank and nothing came back. What's up with that.
# 6  
Old 05-19-2004
print $NR wouldn't have the expected result, it would print the field at the position referenced by the Number of Records (NR) variable, which means that if you've more records than fields, you'll get nothing. For example if there were 6 records, then NR would be set to 6, thus "print $NR" would evaluate to "print $6" hence the sixth field would be printed.

The shortest version of this is (from the command line):

Code:
awk 'BEGIN {RS="===*\n";FS="\n"}{result=$0} END {print result}' my_log_file

........ I think.......

Cheers
ZB
# 7  
Old 05-19-2004
Close again,

The problem seems to be defining a RS. Currently, a single = creates a RS.
this is the bottom of the file:
>>>>>>>>>>>>>>>>>>>>>>>>>
R3V45(66)->launch: debug: /apps/autosys/autosys/bin/sendevent -S TST -E SET_GLOBAL -G 'TSAPNOHO_C_072R_CFI_CONCFI=succeeded'
R3V45(67)->adapter_fork: debug: pids: adapter=6189098, child_task=5316686

R3V45(68)->adapter: exiting with status 0 at Wed May 19 14:33:04 2004


R3V45(69)->adapter_log_#2: debug: Flushing buffers and exiting with _exit(0)
<<<<<<<<<<<<<<<<<<<<<<<<<<
this is the value of result:
>>>>>>>>>>>>>>>>>>>>>>>
5316686

R3V45(68)->adapter: exiting with status 0 at Wed May 19 14:33:04 2004


R3V45(69)->adapter_log_#2: debug: Flushing buffers and exiting with _exit(0)
<<<<<<<<<<<<<<<<<<<<<<<<

Any more ideas?
Thanks,
Kathy
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Logfile monitoring with logfile replacement

Bonjour, I've wrote a script to monitor a logfile in realtime. It is working almost perfeclty except for two things. The script use the following technique : tail -fn0 $logfile | \ while read line ; do ... some stuff done First one, I'd like a way to end the monitoring script if a... (3 Replies)
Discussion started by: Warluck
3 Replies

2. Shell Programming and Scripting

Getting last section of data from logfile

Hi, I have a log file from Munin like this:2012/12/04 13:45:31 : Munin-update finished (29.68 sec) 2012/12/04 13:50:01 Opened log file 2012/12/04 13:50:01 : Starting munin-update 2012/12/04 13:50:01 Error occured in under in the configuration. 2012/12/04 13:50:01 Could not parse datafile... (9 Replies)
Discussion started by: Jotne
9 Replies

3. Shell Programming and Scripting

Prepend first line of section to each line until the next section header

I have searched in a variety of ways in a variety of places but have come up empty. I would like to prepend a portion of a section header to each following line until the next section header. I have been using sed for most things up until now but I'd go for a solution in just about anything--... (7 Replies)
Discussion started by: pagrus
7 Replies

4. Shell Programming and Scripting

how to get one particular section (using awk)?

Hey, I have a problem about how to get one section of a file? I'm new to shell, but by reading some tutorial, I think I can use awk to do this. my input file: >ref|ZP_04937576.1| ECRINAEDPKTFMPSPGKVKHFHAPGGNGVRVDSHLYSGYSVPPNYDSLVGKVITYGAD DEALARMRNALDELIVDGIKTNTELHKDLVRDAAFCKGGVNIHYLE... (11 Replies)
Discussion started by: ritacc
11 Replies

5. Shell Programming and Scripting

Extract section of file based on word in section

I have a list of Servers in no particular order as follows: virtualMachines="IIBSBS IIBVICDMS01 IIBVICMA01"And I am generating some output from a pre-existing script that gives me the following (this is a sample output selection). 9/17/2010 8:00:05 PM: Normal backup using VDRBACKUPS... (2 Replies)
Discussion started by: jelloir
2 Replies

6. Post Here to Contact Site Administrators and Moderators

New section

Hi Just a thought if it already hasn't been suggested. While looking at the forums I thought it might be a good idea under somewhere like 'special forums' add a section called 'projects'. I think this would be good for people to be able to post projects they have created. For example I am... (3 Replies)
Discussion started by: woofie
3 Replies

7. Shell Programming and Scripting

logfile

hi iam new of the ksh script.iwant in formation of how to call in logfile in ksh scripts. if the meaning in ksh. please help me thanks naveen.g (1 Reply)
Discussion started by: naveeng.81
1 Replies

8. Shell Programming and Scripting

what is the most effective way to process a large logfile?

I am dealing with a very large firewall logfile (more than 10G), the logfile like this *snip* Nov 9 10:12:01 testfirewall root: Nov 9 10:12:01 testfirewall root: 0:00:11 accept testfw01-hme0 >hme0 proto: icmp; src: test001.example.net; dst: abc.dst.net; rule: 1; icmp-type: 8;... (8 Replies)
Discussion started by: fedora
8 Replies

9. Post Here to Contact Site Administrators and Moderators

New Section

Just like we have a section "Unix for dummies..." , why not have a section on UNIX BACKUP AND RECOVERY Thanks :) (3 Replies)
Discussion started by: kapilv
3 Replies

10. Post Here to Contact Site Administrators and Moderators

New section

What are the chances of possibly posting some of the more common how-to type stuff for the newbies so we can avoid the repititious stuff that appears every other day? Not so much like a Q&A forum, but more like a reference area for the mundane stuff. Beyond searching the forum, I think people... (3 Replies)
Discussion started by: ober5861
3 Replies
Login or Register to Ask a Question