Logfile - extracting certain lines to concatenate into 1 line Post: 302298524

Sponsored Content

Top Forums Shell Programming and Scripting Logfile - extracting certain lines to concatenate into 1 line Post 302298524 by BecTech on Tuesday 17th of March 2009 08:01:40 PM

03-17-2009

Registered User

Logfile - extracting certain lines to concatenate into 1 line

I've got a log file from automatic diagnostic runs. The log file is appended to each time an automatic log is run.
I'd like to just pull certain lines from each run in the log file, and concatenate them into 1 comma delimited line (for export into excel or an html table).

Each diagnostic run is bracketed by a begin and end comment so I'm able to group it that way but I'm having trouble getting everything into 1 line.

Typical data:

Diags begin.
Tues March 17 18:07:34 EDT 2009

PASS: (123) Power Check
.
.
[more data]
Diags end.
Diags begin.
Tues March 17 19:09:22 EDT 2009

FAIL: (123) Power Check
.
description: Voltage clamp
.
.[more failing data details]
.
Diags end.

I want the timestamp, test result (for the 123 test)

I was able to use awk to get close but I can't quite get it.

awk '/Diags begin/{getline;print};{if ($2=="(123)") print $1,$2,$3,$4}'

This gives me:

Mon Mar 16 11:37:07 EDT 2009
PASS: (123) Power Check
Mon Mar 16 12:31:10 EDT 2009
PASS: (123) Power Check
Tue Mar 17 01:30:54 EDT 2009
**FAIL: (123) Power Check
Tue Mar 17 03:08:16 EDT 2009
PASS: (123) Power Check

What I'm trying to get is:
Mon Mar 16 11:37:07 EDT 2009,PASS: (123) Power Check
Mon Mar 16 12:31:10 EDT 2009,PASS: (123) Power Check
Tue Mar 17 01:30:54 EDT 2009,**FAIL: (123) Power Check
Tue Mar 17 03:08:16 EDT 2009,PASS: (123) Power Check

Also, I'm looking for a way to just pull the information for a certain time frame (for instance the current date - 7 days) but I'll worry about that later.. baby steps... Smilie

The system is Unix (HP-UX) so awk, perl, or sed are options.

Thanks for any help..

Paul

BecTech

View Public Profile for BecTech

Find all posts by BecTech

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extracting Logfile Entries

Hello All, I am trying to write a KSH script that would extract lines from a logfile, within a certain timeframe, in reverse order. Ex. -Sample logfile- INFO 2006-07-12 00:23:00 - ModeDecision Time in WF:27 INFO 2006-07-12 00:24:00 - NRT_MQ_RES:108 INFO 2006-07-12 00:25:00 - getRecords()...

2. UNIX for Advanced & Expert Users

Need solution concatenate and display 2 lines as 1 with a condition for 2 line ?

I have 2 pattern of lines (SQL query and Time taken)in a log i need to capture all SQL queries with time taken >20 sec and need to display as one line. 2 lines from log: 2007-10-23 11:39:17,061 DEBUG - SQL Query : SELECT A.GROUP_CD , C.FN_CD FROM UP_GROUP A , PRD_GROUP_TO_FN B , PRD_FN...

3. Shell Programming and Scripting

concatenate and display 2 lines as 1 with a condition for 2 line ?

4. UNIX for Advanced & Expert Users

need to concatenate two lines if the line doesnt end with quotes

Hi I am getting a source file where the columns are seperated by comma and double Quotes Eg1 : "AAA","BBB","CCCC" in the same source file i am also getting few lines where my last columns double quotes are ending in its next line or the next next line Eg2: "AAA","BBB","CCC CC"...

5. Shell Programming and Scripting

sed / awk to concatenate lines until blank line

Sample input (line feed indicated by ) --------------- The red fox jumped over the brown fence of the red hous He then went into the orchard --------------- Desired Output --------------- The red fox jumped over the brown fence of the red house He then went into the orchard

6. Shell Programming and Scripting

Extracting line matching a phrase and then the next lines after it

Hi all, I was wondering if someone could tell me a way to extract from a file lines where you search for a phrase and then also extract the next X lines after it (i.e. take a block of text from the file)? Example { id=123 time=10:00:00 date=12/12/09 { ........ ...

7. Shell Programming and Scripting

extracting data froma logfile

You are given a 1 year logfile with each line starting with a date in the form �YYYY-MM-DD�. How would you extract logs from the 4th day of each month and put them into a new file

8. Shell Programming and Scripting

Extracting lines after nth LINE from an output

Hi all, Here is my problem for which i am breaking my head for past three days.. I have parted command output as follows.. Model: ATA WDC WD5000AAKS-0 (scsi) Disk /dev/sdb: 500GB Sector size (logical/physical): 512B/512B Partition Table: msdos Number Start End Size Type ...

9. UNIX for Dummies Questions & Answers

Extracting the two lines where the first line is matched

Hi, If I have a file of something like @hg19_gold_AL122127.6-131160 GCTTCATCATGCATGGATAGGCTGGCGCCTTTCCTGAGGCCATATGCCGATGGATATG @hg19_gold_AL122127.6-131159 CTTTAATATTTCCGCCACCATCCTGAGTGAATCCCAGCAAGGACAGTCTTTGGGGATT @hg19_gold_AL122127.6-131158...

10. UNIX for Beginners Questions & Answers

Search for word in huge logfile and need to continue to print few lines from that line til find date

Guys i need an idea for one logic..in shell scripting am struggling with a logic...So the thing is... i need to search for a word in a huge log file and i need to continue to print few more lines from that line and the consecutive line has to end when it finds the line with date..because i know...

LEARN ABOUT OSF1

log

log(8)							      System Manager's Manual							    log(8)

NAME

       log - Records input and output from a program

SYNOPSIS

       /usr/sbin/log <logfile> <command>

OPERANDS

       The file in which to record the interaction being logged.  The command to execute.

DESCRIPTION

       The  log  program  runs <command> and logs the input to and output from <command> to the <logfile> file.  Input and output are logged until
       <command> exits, the log program exits, and the exit status of <command> is returned.

       The log program is used by the system  installation  procedure  and  the  it(8)	command  to  create  the  /var/adm/smlogs/install.log  and
       /var/adm/smlogs/it.log installation log files.

RESTRICTIONS

       Because	the  log  program is used in the installation standalone environment, program size was the greatest concern in its implementation.
       The log program does not search for the PATH variable to locate <command> and error messages are terse.

       The log program causes <command> to take standard input from and write standard output and standard error to  UNIX  pipes.   Some  commands
       will  not  be able to operate in this environment; therefore, it is suggested that you use the script(1) command instead.  UNIX shells will
       not issue prompts when run from log unless the shell is started with an explicit interactive switch (-i for most shells).  For example, log
       foo.tmp /sbin/sh -i

       In the previous example, foo.tmp is the name of <logfile>.

       The  log  program  intercepts  end-of-file (usually Ctrl/d).  Therefore programs which normally receive end-of-file as an exit command must
       exit by some other means.

ERRORS

       Log open error

   Explanation:
       The log program was unable to open <logfile>. Verify that the directory exists and that ownerships and permissions are set correctly.

       Exec Error

   Explanation:
       The log program was unable to execute <command>. Verify that you specified a full pathname for <command> and  that  <command>  is  an  exe-
       cutable file.

       Fork Error

   Explanation:
       The log program was unable to create one of the processes it requires to log data.

SEE ALSO

       Commands: it(8), script(1)

																	    log(8)