Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

replace newline between strings

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting replace newline between strings
# 1  
Old 02-12-2011
replace newline between strings
I'm trying to figure out a way to replace newlines after a unique string is read, all the way until that string occurs again.

The input file looks something like this:
Code:
02/11/11 22:48:00 This is the first line
This is the second line
This is the third line
02/11/11 22:49:00 This is the fifth line
This is the sixth line
02/11/11 22:49:00 This is the seventh line
02/11/11 22:50:00 This is the eighth line

And so on. In this case, the unique string is the date and time strings together.

I'm hoping after replacing the newline characters, it ends up looking like this:
Code:
02/11/11 22:48:00 This is the first line This is the second line This is the third line
02/11/11 22:49:00 This is the fifth line This is the sixth line
02/11/11 22:49:00 This is the seventh line
02/11/11 22:50:00 This is the eighth line

I have been trying something like this, but I don't know what the newline substitution should look like, or if this is even valid because it doesn't work:
Code:
cat file.txt | awk '/^[0-1][0-9]\/[0-9][0-9]\/[0-9][0-9]\ [0-2][0-9]:[0-5][0-9]:[0-5][0-9]/,
     {gsub ("\n"," ")} /[0-1][0-9]\/[0-9][0-9]\/[0-9][0-9]\ [0-2][0-9]:[0-5][0-9]:[0-5][0-9]/' > newfile.txt

Again, in general, I'm trying to read a unique string, then replace all newlines up until the last one encountered before reading that unique string again.

Thanks for any help!
Last edited by Scott; 02-12-2011 at 10:40 AM.. Reason: Please use code tags
# 2  
Old 02-12-2011
Something like this,
Code:
awk '{if(/^[0-9][0-9]\//){printf NR==1?$0:"\n"$0} else{printf FS,$0,FS}}' inputfile

# 3  
Old 02-12-2011
Code:
awk '/^T/{printf $0}/^[0-9]/{printf "\n"$0}' FILE

# 4  
Old 02-12-2011
Code:
awk '{printf (/^[0-9][0-9]\//?RS:FS) $0}' infile

# 5  
Old 02-12-2011
Quote:
Originally Posted by rdcwayx
Code:
awk '{printf (/^[0-9][0-9]\//?RS:FS) $0}' infile

Thanks, this almost worked. I got an error on one line and it bailed. I changed the match part to really match the entire thing I want:

Code:
awk '{printf (/^[0-1][0-9]\/[0-3][0-9]\/[0-9][0-9]\ [0-2][0-9]:[0-5][0-9]:[0-5][0-9]/?RS:FS) $0}'

I'm doing this on a Mac if that makes any difference. Here's the error:
Code:
awk: weird printf conversion % (S
 input record number 57553, file 
 source line number 1
awk: not enough args in printf( compressed by: 0% (SESSION: 123083))
 input record number 57553, file 
 source line number 1

This is a Tivoli activity log, and it might have some weird stuff on the line. The actual lines look something like this:
Code:
01/08/11 19:01:17 ANR00403I Session 123117 ended for node NODE (WinNT).
(SESSION: 123117)
01/08/11 19:01:17 ANE04968I (Session: 123083, Node: NODE) Objects
compressed by: 0% (SESSION: 123083)

(changed slightly so I don't muddy up someone's search results with this nonsense Smilie )

There could be 2, 3, 4 or 5 lines before the date/time stamp occurs again at the beginning of the line.

I'm guessing the logic needs to be a bit more sturdy, something like:

"Find this unique string at the beginning of a line. No matter what, keep reading and keep replacing newlines until you reach this exact sequence again, and then don't replace the last newline encountered."

As long as it doesn't find that uniqe string of a date and timestamp together at the beginning of the line, the above logic should work.

Thanks again, we're getting there.
Last edited by Scott; 02-12-2011 at 11:41 AM.. Reason: Please use code tags for code tags for code, files, data and terminal output
# 6  
Old 02-12-2011
IMO in most cases it is better not to use the formatting part of printf for data. Try this:
Code:
awk '{printf (/^[0-1][0-9]\/[0-3][0-9]\/[0-9][0-9]\ [0-2][0-9]:[0-5][0-9]:[0-5][0-9]/?RS:FS)"%s", $0}' infile

alternatively you may not need that much precision and I would suspect this might be enough:
Code:
awk '{printf (/^..\/..\/.. ..:..:../?RS:FS)"%s", $0}' infile

Last edited by Scrutinizer; 02-12-2011 at 12:53 PM..
This User Gave Thanks to Scrutinizer For This Post:
acruhl
# 7  
Old 02-12-2011
Quote:
Originally Posted by Scrutinizer
IMO in most cases it is not good practice to use the formatting part of printf for data. Try this:
Code:
awk '{printf (/^[0-1][0-9]\/[0-3][0-9]\/[0-9][0-9]\ [0-2][0-9]:[0-5][0-9]:[0-5][0-9]/?RS:FS)"%s", $0}' infile

alternatively you may not need that much precision and I would suspect this might be enough:
Code:
awk '{printf (/^..\/..\/.. ..:..:../?RS:FS)"%s", $0}' infile

I tried the first one and it worked with a small modification: to match the first 2 characters in the next field. Apparently some of my lines start with a date and time stamp but aren't really what I want to match.

If you don't mind explaining, why not use such specific syntax? Performance maybe? I could see why your second example might be quicker. Well, I might as well test it:
Code:
$ time cat file.txt | awk '{printf (/^[0-1][0-9]\/[0-3][0-9]\/[0-9][0-9]\ [0-2][0-9]:[0-5][0-9]:[0-5][0-9]\ AN/?RS:FS)"%s", $0}' > temp.out

real    0m1.447s
user    0m1.251s
sys    0m0.126s

$ time cat file.txt | awk '{printf (/^..\/..\/.. ..:..:..\ AN/?RS:FS)"%s", $0}' > temp.out
real    0m1.498s
user    0m1.257s
sys    0m0.128s

Hmm. Your second syntax does work exactly the same given that I add the "\ AN" part to both.

Thanks!

---------- Post updated at 08:40 AM ---------- Previous update was at 08:39 AM ----------

By the way, both of your examples were exactly what I needed after I made the small change. Thanks!
Last edited by Scott; 02-12-2011 at 11:45 AM.. Reason: Hmm. Code tags :)
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Replace newline in a string

I have a string like below: {\rtf1\fbidis\ansi\deff0{\fonttbl{\f0\fnil\fcharset0 MS Sans Serif;}{\f1\fnil MS Sans Serif;}} \viewkind4\uc1\pard\ltrpar\lang2057\f0\fs16 19/11/2010 SOME DESCRIPTION. \par \lang1033\f1\par } I have to replace the newline character with null in the... (8 Replies)
Discussion started by: Pratik4891
8 Replies

2. Shell Programming and Scripting

Replace String With Newline

Hi, I'm struggling with a string replacement. I have an XML file which is in the following layout <FUNCTION> <PRODUCTS> <PRODUCT CODE="PRODUCE" ACTION="amend" VALIDATE="no"> <SUPPLIER PRODUCT="PRODUCT" ACTION="amend" CODE="SUPPLIER"> <STOCK_QUANTITY DATA="21"/> ... (15 Replies)
Discussion started by: Ste_Moore01
15 Replies

3. Shell Programming and Scripting

any savant ? using AWK/SED to remove newline character between two strings : conditional removal

I'd like to remove (do a pattern or precise replacement - this I can handle in SED using Regex ) ---AFTER THE 1ST Occurrence ( i.e. on the 2nd occurrence - from the 2nd to fourth occurance ) of a specific string : type 1 -- After the 1st occurrence of 1 string1 till the 1st occurrence of... (4 Replies)
Discussion started by: sieger007
4 Replies

4. Shell Programming and Scripting

Replace newline with comma.

I have output from a file like this: 15,01,11,14:06 235 I would like to change this to: 15,01,11,14:06,235 Removing newline and change to "," I now this can be done with tr cat OUT | tr '\n' ','' My problem is that tr is not implemented in this shell. sed is, show it should be... (7 Replies)
Discussion started by: Jotne
7 Replies

5. Shell Programming and Scripting

replace >< with > newline <

Hi All, I have the command in PERL for performing this, but Can you please suggest me how can i perform this using AWK: My input xml file looks like this: <aaa>hello</aaa><bbb>hai</bbb> I want the output like this ( means need new line after end of each xml tag): <aaa>hello</aaa>... (1 Reply)
Discussion started by: HemaV
1 Replies

6. Shell Programming and Scripting

Replace a string with newline

Hi all I have the problem to substitute a string with newline in Perl. Can anybody help me? And also how to replace a string with opening bracket (e.g. (START ) with a whitespace/null character? Thanks in advance. (1 Reply)
Discussion started by: my_Perl
1 Replies

7. UNIX for Dummies Questions & Answers

how to replace a text with a newline

Hi. How to replace a text in a file with starting from a newline example==== shashi how r u ? shashi , r u their? shashi.... shashi hello shashi output will be shashi how r u ? shashi , r u their? shashi.... shashi hello shashi ========= thanks in advance.. (3 Replies)
Discussion started by: hegdeshashi
3 Replies

8. Shell Programming and Scripting

Replace comma with newline

Hi, for some reason I cant seem to figure this out. I have a file which looks something like this word word word word word,word,word word word word,word,word,word,word word word Basically I want this whole thing to be a list with 1 word on each line like this... word word word... (1 Reply)
Discussion started by: eltinator
1 Replies

9. Shell Programming and Scripting

replace a newline (\n)

dear all: maybe i have a file like : 12 34 56 78 end how do write can i replace newline into NA : make the file inte : 12 NA 34 NA 56 78 END (3 Replies)
Discussion started by: jeter
3 Replies

10. UNIX for Dummies Questions & Answers

How can I replace newline character?

Hi, I am trying to write a script to prepare some text for use as web content. What is happening is that all the newlines in the textfile are ignored, so I want to be able to replace/add a few characters so that for a file containg: This is line 1. This is line two. This is line four.... (1 Reply)
Discussion started by: ghoti
1 Replies
Login or Register to Ask a Question