Merge multiple lines into a single line


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Merge multiple lines into a single line
# 8  
Old 07-04-2017
Thanks Don. I used the command line below. Right behind each semi-colon is a \r following by what looks like a space.

Code:
awk 'sub (/;$/, "")+1 && $1=$1' RS= infile
0000000 357 273 277 164 151 155 145 163 164 141 155 160 075 062 060 061
        357 273 277   t   i   m   e   s   t   a   m   p   =   2   0   1
0000020 067 055 060 066 055 062 070 055 060 061 056 060 061 056 063 065
          7   -   0   6   -   2   8   -   0   1   .   0   1   .   3   5
0000040 056 060 070 060 065 067 066 073 015 040 145 166 145 156 164 040
          .   0   8   0   5   7   6   ;  \r       e   v   e   n   t    
0000060 163 164 141 164 165 163 075 060 073 015 040 165 163 145 162 151
          s   t   a   t   u   s   =   0   ;  \r       u   s   e   r   i
0000100 144 075 165 163 145 162 061 073 015 040 141 165 164 150 151 144
          d   =   u   s   e   r   1   ;  \r       a   u   t   h   i   d
0000120 075 165 163 145 162 061 073 015 040 141 160 160 154 151 143 141
          =   u   s   e   r   1   ;  \r       a   p   p   l   i   c   a
0000140 164 151 157 156 040 151 144 075 061 060 056 061 060 056 061 060
          t   i   o   n       i   d   =   1   0   .   1   0   .   1   0
0000160 056 061 060 056 061 061 061 061 061 056 061 062 063 064 065 066
          .   1   0   .   1   1   1   1   1   .   1   2   3   4   5   6
0000200 067 070 071 060 061 073 015 040 141 160 160 154 151 143 141 164
          7   8   9   0   1   ;  \r       a   p   p   l   i   c   a   t
0000220 151 157 156 040 156 141 155 145 075 107 125 111 073 015 040 015
          i   o   n       n   a   m   e   =   G   U   I   ;  \r      \r
0000240 040 164 151 155 145 163 164 141 155 160 075 062 060 061 067 055
              t   i   m   e   s   t   a   m   p   =   2   0   1   7   -
0000260 060 066 055 062 070 055 060 061 056 060 061 056 063 066 056 060
          0   6   -   2   8   -   0   1   .   0   1   .   3   6   .   0
0000300 071 066 064 070 066 073 015 040 145 166 145 156 164 040 163 164
          9   6   4   8   6   ;  \r       e   v   e   n   t       s   t
0000320 141 164 165 163 075 060 073 015 040 165 163 145 162 151 144 075
          a   t   u   s   =   0   ;  \r       u   s   e   r   i   d   =
0000340 165 163 145 162 061 073 015 040 141 165 164 150 151 144 075 165
          u   s   e   r   1   ;  \r       a   u   t   h   i   d   =   u
0000360 163 145 162 061 073 015 040 141 160 160 154 151 143 141 164 151
          s   e   r   1   ;  \r       a   p   p   l   i   c   a   t   i
0000400 157 156 040 151 144 075 061 060 056 061 060 056 061 060 056 061
          o   n       i   d   =   1   0   .   1   0   .   1   0   .   1
0000420 060 056 061 061 061 061 061 056 061 062 063 064 065 066 067 070
          0   .   1   1   1   1   1   .   1   2   3   4   5   6   7   8
0000440 071 060 061 073 015 040 141 160 160 154 151 143 141 164 151 157
          9   0   1   ;  \r       a   p   p   l   i   c   a   t   i   o
0000460 156 040 156 141 155 145 075 107 125 111 073 015 040 163 164 141
          n       n   a   m   e   =   G   U   I   ;  \r       s   t   a
0000500 164 145 155 145 156 164 040 164 145 170 164 075 123 105 114 105
          t   e   m   e   n   t       t   e   x   t   =   S   E   L   E
0000520 103 124 040 164 141 142 154 145 056 146 151 145 154 144 054 040
          C   T       t   a   b   l   e   .   f   i   e   l   d   ,    
0000540 164 141 142 154 145 056 146 151 145 154 144 054 040 164 141 142
          t   a   b   l   e   .   f   i   e   l   d   ,       t   a   b
0000560 154 145 056 146 151 145 154 144 040 146 162 157 155 040 144 141
          l   e   .   f   i   e   l   d       f   r   o   m       d   a
0000600 164 141 142 141 163 145 040 167 150 145 162 145 040 164 141 142
          t   a   b   a   s   e       w   h   e   r   e       t   a   b
0000620 154 145 056 146 151 145 154 144 040 075 040 166 141 154 165 145
          l   e   .   f   i   e   l   d       =       v   a   l   u   e
0000640 015 040 015 012
         \r      \r  \n
0000644

# 9  
Old 07-04-2017
Try
Code:
awk 'gsub (";$|\r", "")+1 && $1=$1' RS= file

This User Gave Thanks to RudiC For This Post:
# 10  
Old 07-04-2017
Thanks RudiC. We are getting closer thanks to you all. The last command put everything on one line. The events should all begin with the timestamp field. So by using the example I provided there should be 2 output lines.
# 11  
Old 07-04-2017
You seem not to have any <line feed> chars in your file except in the very end. So mayhap the condition you quoted in post#1 - empty line between records - can't be met with non- *nix files. Is that a MAC file?
This User Gave Thanks to RudiC For This Post:
# 12  
Old 07-04-2017
Quote:
Originally Posted by RudiC
You seem not to have any <line feed> chars in your file except in the very end. So mayhap the condition you quoted in post#1 - empty line between records - can't be met with non- *nix files. Is that a MAC file?
We haven't seen enough information to determine whether or not there are <newline> (or <linefeed>) characters in the input file. Using RS= in awk to set the record separator to sequences of blank lines will not treat a line containing a <carriage-return> as a blank line. So, awk won't find any record separators in a DOS input file. (And, if you remove the <carriage-return> characters in the code that will happen AFTER the record separator search has already occurred.)

I would start by trying:
Code:
awk '
function p() {
	if(d) {
		sub(/;$/, "", o)
		print o
		o = ""
		d = 0
	}
}
{	sub(/\r/, "")
}
!NF {	p()
	next
}
{	$1 = $1
	o = o (o == "" ? "" : " ") $0
	d = 1
}
END {	p()
}' file

If that doesn't work, please show us the output from:
Code:
od -bc file

where file is the name of your input file.
This User Gave Thanks to Don Cragun For This Post:
# 13  
Old 07-05-2017
Hmmm - in the od output in post#8, I find many \r but just one \n. So my question... I only now see the three introductory bytes 357 273 277 - need to double check their meaning...
This User Gave Thanks to RudiC For This Post:
# 14  
Old 07-05-2017
I think the od dump is from your first awk's output that was all on one line.
As Don said, the RS= will not see a \n\r\n but in GNU awk you can set RS="\n\r?\n" or even RS="\n[[:space:]]*\n" to allow any whitespace including \r and further \n.
This User Gave Thanks to MadeInGermany For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Merge multi-lines into one single line using shell script or Linux command

Hi, Can anyone help me for merge the following multi-line log which beginning with a " and line ending with ": into one line. *****Original Log***** 087;2008-12-06;084403;"mc;;SYHLR6AP1D\LNZW;AD-703;1;12475;SYHLR6AP1B;1.1.1.1;0000000062;HGPDI:MSISDN=12345678,APNID=1,EQOSID=365;... (3 Replies)
Discussion started by: rajeshlinux2010
3 Replies

2. UNIX for Dummies Questions & Answers

Need help combining txt files w/ multiple lines into csv single cell - also need data merge

:confused:Hello -- i just joined the forums. I am a complete noob -- only about 1 week into learning how to program anything... and starting with linux. I am working in Linux terminal. I have a folder with a bunch of txt files. Each file has several lines of html code. I want to combine... (2 Replies)
Discussion started by: jetsetter
2 Replies

3. Shell Programming and Scripting

Merge multiple lines in one line

Hi guys, So i have a input file with several sequences aligned (fasta) >NC_005930 241 bp MNMINIFIINNIFDQFIPVKLSIFSLTSVGSIIA LSWVWINTKTHWAISRSNTP-SLLLNSL WTLLITNL-NEKTNPWAPWLFSLFLLCFSFNIMSLI-PYTF-SQ TSHLSFTFGLSLPIWIMVNIAGFKNNWKKKISHLLPQGTPIYLVPVMII IETISLFIQPLTLGFRLGANLLAGHLLIFLCSCTIWE... (6 Replies)
Discussion started by: andreia
6 Replies

4. Shell Programming and Scripting

Merge multiple lines to one line when line starts with and ends with

example: comment Now_TB.table column errac is for error messages 1 - first 2 - second 3 -third ; in this example I need to be able to grab the comment as first word and ; as the last word and it might span a few lines. I need it to be put all in one line without line breaks so I can... (4 Replies)
Discussion started by: wambli
4 Replies

5. Shell Programming and Scripting

Multiple lines in a single column to be merged as a single line for a record

Hi, I have a requirement with, No~Dt~Notes 1~2011/08/1~"aaa bbb ccc ddd eee fff ggg hhh" Single column alone got splitted into multiple lines. I require the output as No~Dt~Notes 1~2011/08/1~"aaa<>bbb<>ccc<>ddd<>eee<>fff<>ggg<>hhh" mean to say those new lines to be... (1 Reply)
Discussion started by: Bhuvaneswari
1 Replies

6. UNIX for Advanced & Expert Users

Merge a group of lines into single line

Hi Everybody, Below are the contents of the a text file .., SN = 8 MSI = 405027002277133 IKVALUE = DE6AA6A11D42B69DF6398D44B17BC6F2 K4SNO = 2 CARDTYPE = SIM ALG = COMP128_3 SN = 8 MSI = 405027002546734 IKVALUE = 1D9F8BAA73973D8FBF8CBFB01436D822 K4SNO = 2 CARDTYPE = SIM ALG =... (8 Replies)
Discussion started by: prasanth_babu
8 Replies

7. Shell Programming and Scripting

merge lines into single line based on symbol \t

The symbols are \t and \t\t (note: not tab) If the line starts with \t merge them into a single line upto symbol \t\t \t\t to end and start new line I able to join in a single line but not ending at \t\t and I completely confused help would be appreciated:b::D Input \ta tab XXXXXXXXXX \te... (5 Replies)
Discussion started by: repinementer
5 Replies

8. Shell Programming and Scripting

Help on Merge multi-lines into one single line

Hello, Can anyone let me know how to use Perl script to Merge following multi-lines into one single line... ***** Multi-line***** FILE_Write root OK Tue Jul 01 00:00:00 2008 cl_get_path file descriptor = 1 FILE_Write root OK ... (5 Replies)
Discussion started by: happyday
5 Replies

9. Shell Programming and Scripting

Merge multi-lines into one single line

Hi, Can anyone help me for merge the following multi-line log which beginning with a number and time: into one line. For each line need to delete the return and add a space. Please see the red color line. *****Original Log*****... (4 Replies)
Discussion started by: happyday
4 Replies

10. Shell Programming and Scripting

Removing end of line to merge multiple lines

I'm sure this will be an easy question for you experts out there, but I have been searching the forum and working on this for a couple hours now and can't get it right. I have a very messy data file that I am trying to tidy up - one of the issues is some records are split into multiple lines: ... (4 Replies)
Discussion started by: tink
4 Replies
Login or Register to Ask a Question