Logfile - extracting certain lines to concatenate into 1 line Post: 302298524

Sponsored Content

Top Forums Shell Programming and Scripting Logfile - extracting certain lines to concatenate into 1 line Post 302298524 by BecTech on Tuesday 17th of March 2009 08:01:40 PM

03-17-2009

Registered User

Logfile - extracting certain lines to concatenate into 1 line

I've got a log file from automatic diagnostic runs. The log file is appended to each time an automatic log is run.
I'd like to just pull certain lines from each run in the log file, and concatenate them into 1 comma delimited line (for export into excel or an html table).

Each diagnostic run is bracketed by a begin and end comment so I'm able to group it that way but I'm having trouble getting everything into 1 line.

Typical data:

Diags begin.
Tues March 17 18:07:34 EDT 2009

PASS: (123) Power Check
.
.
[more data]
Diags end.
Diags begin.
Tues March 17 19:09:22 EDT 2009

FAIL: (123) Power Check
.
description: Voltage clamp
.
.[more failing data details]
.
Diags end.

I want the timestamp, test result (for the 123 test)

I was able to use awk to get close but I can't quite get it.

awk '/Diags begin/{getline;print};{if ($2=="(123)") print $1,$2,$3,$4}'

This gives me:

Mon Mar 16 11:37:07 EDT 2009
PASS: (123) Power Check
Mon Mar 16 12:31:10 EDT 2009
PASS: (123) Power Check
Tue Mar 17 01:30:54 EDT 2009
**FAIL: (123) Power Check
Tue Mar 17 03:08:16 EDT 2009
PASS: (123) Power Check

What I'm trying to get is:
Mon Mar 16 11:37:07 EDT 2009,PASS: (123) Power Check
Mon Mar 16 12:31:10 EDT 2009,PASS: (123) Power Check
Tue Mar 17 01:30:54 EDT 2009,**FAIL: (123) Power Check
Tue Mar 17 03:08:16 EDT 2009,PASS: (123) Power Check

Also, I'm looking for a way to just pull the information for a certain time frame (for instance the current date - 7 days) but I'll worry about that later.. baby steps... Smilie

The system is Unix (HP-UX) so awk, perl, or sed are options.

Thanks for any help..

Paul

BecTech

View Public Profile for BecTech

Find all posts by BecTech

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extracting Logfile Entries

Hello All, I am trying to write a KSH script that would extract lines from a logfile, within a certain timeframe, in reverse order. Ex. -Sample logfile- INFO 2006-07-12 00:23:00 - ModeDecision Time in WF:27 INFO 2006-07-12 00:24:00 - NRT_MQ_RES:108 INFO 2006-07-12 00:25:00 - getRecords()...

2. UNIX for Advanced & Expert Users

Need solution concatenate and display 2 lines as 1 with a condition for 2 line ?

I have 2 pattern of lines (SQL query and Time taken)in a log i need to capture all SQL queries with time taken >20 sec and need to display as one line. 2 lines from log: 2007-10-23 11:39:17,061 DEBUG - SQL Query : SELECT A.GROUP_CD , C.FN_CD FROM UP_GROUP A , PRD_GROUP_TO_FN B , PRD_FN...

3. Shell Programming and Scripting

concatenate and display 2 lines as 1 with a condition for 2 line ?

4. UNIX for Advanced & Expert Users

need to concatenate two lines if the line doesnt end with quotes

Hi I am getting a source file where the columns are seperated by comma and double Quotes Eg1 : "AAA","BBB","CCCC" in the same source file i am also getting few lines where my last columns double quotes are ending in its next line or the next next line Eg2: "AAA","BBB","CCC CC"...

5. Shell Programming and Scripting

sed / awk to concatenate lines until blank line

Sample input (line feed indicated by ) --------------- The red fox jumped over the brown fence of the red hous He then went into the orchard --------------- Desired Output --------------- The red fox jumped over the brown fence of the red house He then went into the orchard

6. Shell Programming and Scripting

Extracting line matching a phrase and then the next lines after it

Hi all, I was wondering if someone could tell me a way to extract from a file lines where you search for a phrase and then also extract the next X lines after it (i.e. take a block of text from the file)? Example { id=123 time=10:00:00 date=12/12/09 { ........ ...

7. Shell Programming and Scripting

extracting data froma logfile

You are given a 1 year logfile with each line starting with a date in the form �YYYY-MM-DD�. How would you extract logs from the 4th day of each month and put them into a new file

8. Shell Programming and Scripting

Extracting lines after nth LINE from an output

Hi all, Here is my problem for which i am breaking my head for past three days.. I have parted command output as follows.. Model: ATA WDC WD5000AAKS-0 (scsi) Disk /dev/sdb: 500GB Sector size (logical/physical): 512B/512B Partition Table: msdos Number Start End Size Type ...

9. UNIX for Dummies Questions & Answers

Extracting the two lines where the first line is matched

Hi, If I have a file of something like @hg19_gold_AL122127.6-131160 GCTTCATCATGCATGGATAGGCTGGCGCCTTTCCTGAGGCCATATGCCGATGGATATG @hg19_gold_AL122127.6-131159 CTTTAATATTTCCGCCACCATCCTGAGTGAATCCCAGCAAGGACAGTCTTTGGGGATT @hg19_gold_AL122127.6-131158...

10. UNIX for Beginners Questions & Answers

Search for word in huge logfile and need to continue to print few lines from that line til find date

Guys i need an idea for one logic..in shell scripting am struggling with a logic...So the thing is... i need to search for a word in a huge log file and i need to continue to print few more lines from that line and the consecutive line has to end when it finds the line with date..because i know...

LEARN ABOUT NETBSD

schedctl

SCHEDCTL(8)						    BSD System Manager's Manual 					       SCHEDCTL(8)

NAME

     schedctl -- control scheduling of processes and threads

SYNOPSIS

     schedctl [-A cpus] [-C class] [-P pri] [-t lid] -p pid | command

DESCRIPTION

     The schedctl command can be used to control the scheduling of processes and threads.  It also returns information about the current schedul-
     ing parameters of the process or thread.  Only the super-user may change the scheduling parameters.  schedctl can also be used to start a new
     command using the specified parameters.

     Available options:

     -A cpus	Set of the processors on which process or thread should run, that is, affinity.  Processors are defined as numbers (starting from
		zero) and separated by commas.	A value of -1 is used to unset the affinity.

     -C class	Scheduling class (policy), one of:

		SCHED_OTHER   Time-sharing (TS) scheduling policy.  The default policy in NetBSD.

		SCHED_FIFO    First in, first out (FIFO) scheduling policy.

		SCHED_RR      Round-robin scheduling policy.

     -P pri	Priority for the process or thread.  Value should be in the range from SCHED_PRI_MIN (0) to SCHED_PRI_MAX (63).  Setting of prior-
		ity for the process or thread running at SCHED_OTHER policy is not allowed.

     -p pid	The target process which will be affected.  If the process has more than one thread, all of them will be affected.

		If -p is not given, a command to execute must be given on the command line.

     -t lid	Thread in the specified process.  If specified, only this thread in the process will be affected.  May only be specified if -p is
		also given.

EXAMPLES

     Show scheduling information about the process whose ID is ``123'':

	   # schedctl -p 123

     Set the affinity to CPU 0 and CPU 1, policy to SCHED_RR, and priority to 63 for thread whose ID is ``1'' in process whose ID is ``123'':

	   # schedctl -p 123 -t 1 -A 0,1 -C SCHED_RR -P 63

     Run the top(1) command with real-time priority:

	   # schedctl -C SCHED_FIFO top

SEE ALSO

     nice(1), getpriority(2), setpriority(2), psrset(8), renice(8)

HISTORY

     The schedctl command first appeared in NetBSD 5.0.

BSD
								  March 21, 2011							       BSD