Sponsored Content
Top Forums Shell Programming and Scripting concatenate log file lines up to timestamp Post 302320707 by AlanC on Thursday 28th of May 2009 04:30:10 PM
Old 05-28-2009
concatenate log file lines up to timestamp

Hi,

Using sed awk or perl I am trying to do something similar to

https://www.unix.com/shell-programmin...nk-line-2.html

but my requirement is slightly different. What I am trying to accomplish is to reformat a logfile such that all lines start with the timestamp line and any lines that do no start with a timestamp are appended to the last line with a timestamp. Optionally I would like to do this up to the first semicolon.

A simplified input would be somthing like this

2009-05-27 02:37:27.283 The quick
brown fox;
The quick
brown fox
2009-05-28 10:10:28.000 Mary
had a
little lamb.
2009-06-01 19:37:29.000 Jack and Jill ran up the hill;

and ideally the output would be

2009-05-27 02:37:27.283 The quick brown fox;
2009-05-28 10:10:28.000 Mary had a little lamb.
2009-06-01 19:37:29.000 Jack and Jill ran up the hill;

although this is also acceptable

2009-05-27 02:37:27.283 The quick brown fox; The quick brown fox
2009-05-28 10:10:28.000 Mary had a little lamb.
2009-06-01 19:37:29.000 Jack and Jill ran up the hill;

The log files can be up to 10MB in size and there can be a hundred lines or more between timestamps. The purpose of this is to format the file so that it can be loaded into a database.

Any suggestions/solutions would be greatly appreciated.

Thanks,
Alan
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Copy lines from a log file based on timestamp

how to copy lines from a log file based on timestamp. INFO (RbrProcessFlifoEventSessionEJB.java:processFlight:274) - E_20080521_110754_967: rbrAciInfoObjects listing complete! INFO (RbrPnrProcessEventSessionEJB.java:processFlight:197) - Event Seq: 1647575217; Carrier: UA; Flt#: 0106; Origin:... (1 Reply)
Discussion started by: ranjiadmin
1 Replies

2. Shell Programming and Scripting

copy lines from log files based on timestamp and sysdate

I am sorry to repost this question. it was not clear, and I had the meeting and didn't response the question on time. I do really need help and appreciate your help very much. I'm looking for a simple shell script that will read lots of audit log file (*.aud) in a log fold every 10 minutes,... (1 Reply)
Discussion started by: percvs88
1 Replies

3. Shell Programming and Scripting

copy lines from log files based on timestamp and sysdate

I'm looking for a command or simple script that will read lots of audit log file (*.aud) in log fold every 10 minutes, and will output to one file based on sysdate - 10 minutes. assume the script is run at 11:12:20, and it should grep the line from Wed Jun 17 11:02:43 2009 to end of file. after... (4 Replies)
Discussion started by: percvs88
4 Replies

4. Shell Programming and Scripting

Delete log file entries based on the Date/Timestamp within log file

If a log file is in the following format 28-Jul-10 ::: Log message 28-Jul-10 ::: Log message 29-Jul-10 ::: Log message 30-Jul-10 ::: Log message 31-Jul-10 ::: Log message 31-Jul-10 ::: Log message 1-Aug-10 ::: Log message 1-Aug-10 ::: Log message 2-Aug-10 ::: Log message 2-Aug-10 :::... (3 Replies)
Discussion started by: vikram3.r
3 Replies

5. Shell Programming and Scripting

Concatenate lines between lines starting with a specific pattern

Hi, I have a file such as: --- >contig00001 length=35524 numreads=2944 gACGCCGCGCGCCGCGGCCAGGGCTGGCCCA CAGGCCGCGCGGCGTCGGCTGGCTGAG >contig00002 length=4242 numreads=43423 ATGCCGAAGGTCCGCCTGGGGCTGG CGCCGGGAGCATGTAGCG --- I would like to concatenate the lines not starting with ">"... (9 Replies)
Discussion started by: s052866
9 Replies

6. Programming

Concatenate two lines in a fIle

Hi All, Can any one help me in finding the solution for concatenating two or more lines in a file and writing them to a temporary file. for Example: He is a wise student. So he got first rank. This is in a file i want the output as He is a wise student so he got first rank. into a file... (3 Replies)
Discussion started by: uday.sena.m
3 Replies

7. UNIX for Advanced & Expert Users

Concatenate lines in file shell script

Hi colleagues, I have a file in this format. "/cccc/pppp/dddd/ggg/prueba.txt". ERROR" THE error bbbbbbbbbb finish rows. "/kkkk/mmmm/hhhh/jjj/ejemplo.txt". ERROR This is other error rows.I need my file in this format. "/cccc/pppp/dddd/ggg/prueba.txt". ERROR" THE error bbbbbbbbbb finish rows.... (3 Replies)
Discussion started by: systemoper
3 Replies

8. Shell Programming and Scripting

Check/Parse log file's lines using time difference/timestamp

I was looking at this script which outputs the two lines which differs less than one sec. #!/usr/bin/perl -w use strict; use warnings; use Time::Local; use constant SEC_MILIC => 1000; my $file='infile'; ## Open for reading argument file. open my $fh, "<", $file or die "Cannot... (1 Reply)
Discussion started by: cele_82
1 Replies

9. Shell Programming and Scripting

To check timestamp in logfile and display lines upto 3 hours before current timestamp

Hi Friends, I have the following logfile. Currently time in india is 07/31/2014 12:33:34 and i have the following content in logfile. I want to display only those entries which contain string 'Exception' within last 3 hours. In this case, it would be the last line only I can get the... (12 Replies)
Discussion started by: srkmish
12 Replies

10. Shell Programming and Scripting

Grep lines between last hour timestamp and current timestamp

So basically I have a log file and each line in this log file starts with a timestamp: MON DD HH:MM:SS SEP 15 07:30:01 I need to grep all the lines between last hour timestamp and current timestamp. Then these lines will be moved to a tmp file from which I will grep for particular strings. ... (1 Reply)
Discussion started by: nms
1 Replies
DtEditorInsert(library call)											      DtEditorInsert(library call)

NAME
DtEditorInsert -- insert data into a DtEditor widget SYNOPSIS
#include <Dt/Editor.h> DtEditorErrorCode DtEditorInsert( Widget widget, DtEditorContentRec *data); DESCRIPTION
The DtEditorInsert function inserts either a string, wide character string or sized buffer at the insertion cursor position in a DtEditor widget. The data is transferred to the DtEditor widget using a DtEditorContentRec, which indicates the type of data being transferred along with the actual data. After the data is appended, the insertion cursor is positioned after the last character inserted. The widget argument specifies the DtEditor widget ID. The data argument is a pointer to a data structure containing the data to insert. For a complete definition of the DtEditor widget and its associated resources, see DtEditor(3). For a complete definition of DtEditorContentRec, see Dt/Editor.h - DtEditor(5). RETURN VALUE
Upon successful completion, the DtEditorInsert function returns one of the following values: DtEDITOR_NO_ERRORS The data was inserted sucessfully. DtEDITOR_NULLS_REMOVED NULL characters were found and removed from the data. Otherwise, if the DtEditorInsert function cannot insert the data into the DtEditor widget, it returns one of the following values: DtEDITOR_INVALID_TYPE The Type field is not recognized. DtEDITOR_ILLEGAL_SIZE The size of the buffer passed in is negative. DtEDITOR_NULL_ITEM The buffer is NULL. DtEDITOR_INSUFFICIENT_MEMORY Not enough system memory is available to insert the data. EXAMPLES
The following code segment sets the contents of a DtEditor widget to ``The quick brown fox.'' Widget editor; DtEditorContentRec cr; DtEditorErrorCode status; char *sampleString1="The brown fox", *sampleString2=" quick"; cr.type = DtEDITOR_TEXT; cr.value.string = sampleString1; status = DtEditorSetContents(editor, &cr); if (status != DtEDITOR_NO_ERRORS && status != DtEDITOR_NULLS_REMOVED) { printf("Unable to set contents of the widget "); } else { /* * Move the insertion cursor so it is after the * letter 'e' in "The". */ DtEditorSetInsertionCursorPosition(editor, 2); cr.type = DtEDITOR_TEXT; cr.data.string = sampleString2; status = DtEditorInsert(editor, &cr); if (status != DtEDITOR_NO_ERRORS) printf("Unable to insert into the contents of the widget "); } APPLICATION USAGE
If the data is in a disk file, rather than in memory, the application should use DtEditorInsertFromFile(3). SEE ALSO
Dt/Editor.h - DtEditor(5), DtEditor(3), DtEditorAppend(3), DtEditorAppendFromFile(3), DtEditorGetContents(3), DtEditorInsertFromFile(3), DtEditorReplace(3), DtEditorReplaceFromFile(3), DtEditorSaveContentsToFile(3), DtEditorSetContents(3), DtEditorGetInsertionPosition(3), DtEditorSetInsertionPosition(3), DtEditorSetContentsFromFile(3). DtEditorInsert(library call)
All times are GMT -4. The time now is 09:19 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy